In a talk at AI Engineer 2025, Eugene Yan argues that recommender systems and search are converging with large language models. He highlights three techniques driving this shift: semantic IDs, data augmentation, and unified foundation models.
Semantic IDs map items into a representation that LLMs can process, allowing recommendation tasks to be framed more like language tasks. Data augmentation, meanwhile, uses LLM-generated content to fill gaps in user-item data, which is especially useful when interactions are sparse.
The third direction is unified foundation models that combine retrieval and ranking in a single system. Yan presents these as practical levers for improving existing RecSys and search pipelines rather than as purely theoretical ideas.