In a recent blog post, Eugene Yan documents a series of experiments using large language models to help with research, reflection, and planning. The experiments suggest that LLMs are useful for synthesizing information and structuring thought, but they also reveal a bottleneck: getting the right documents into the model's context. Without reliable access to relevant material, the quality of the LLM's output degrades quickly.
The author identifies document retrieval as a major shortcoming in these workflows. Simple retrieval methods often fail to surface the most useful sources, which limits how effectively an LLM can reason about a topic. Yan argues that techniques from search and recommender systems—such as improved ranking, embeddings, and candidate generation—can be adapted to strengthen the retrieval step and make LLM-assisted research more robust.
The post is a practical exploration rather than a formal study, but it highlights an important pattern: as LLMs move into personal productivity tools, the surrounding infrastructure for finding and selecting information becomes critical. Yan's experiments point toward a hybrid approach, combining the reasoning power of LLMs with established retrieval and recommendation methods to get the best of both worlds.