In his AI Engineer 2023 keynote, Eugene Yan lays out the core building blocks he sees as essential for LLM systems. The talk centers on four components: evaluations, retrieval-augmented generation, guardrails, and collecting feedback. Yan presents these not as optional extras but as the fundamental scaffolding for building reliable systems on top of large language models.
The emphasis on evals suggests that measuring model performance is where effective LLM engineering starts. Retrieval-augmented generation, meanwhile, addresses the need to connect models to up-to-date or domain-specific information that they were not trained on.
Yan rounds out the framework with guardrails—mechanisms to keep outputs within acceptable bounds—and systematic feedback collection to drive improvement over time. As a single-source view, the talk offers a concise checklist for teams designing LLM applications, though it does not go into implementation detail in the abstract itself.