A new arXiv preprint introduces HERO, short for Harnessing Evidence for Test-time Refinement and Prompt Optimization in Program Generation. The method is designed to improve LLM performance on program generation tasks such as text-to-SQL and text-to-Python by refining reusable prompts and scaffolding during inference.

The paper argues that while harness optimization has shown promise for improving LLMs, it needs further adaptation for program generation. Existing methods, the authors note, often depend on LLM reflectors to guide refinement. HERO instead centers on using evidence at test time to guide prompt and scaffolding updates.

Because the abstract is truncated, full implementation details and evaluation results are not yet available from the listing. Still, the proposed direction points toward more evidence-driven, reusable scaffolding for code-generation benchmarks.