New benchmark tests medical ML agents beyond raw accuracy
Auto-MD-bench evaluates autonomous machine-learning agents on real-world medical tasks, arguing that strong metric scores alone don't ensure practical usefulness.
Research Digest
Preprints and papers, rewritten in plain English.
Can LLM Proxies Replace Humans in AI-Reliance Studies?
A new arXiv preprint questions whether simulated users can validly stand in for humans when measuring reliance on AI answers.
ContextCompact 2.0 Aims to Keep Small-Language-Model Agents on Track
A new arXiv preprint describes a context-compaction technique to help small language models sustain long agent sessions despite constrained context windows.
Learning to Pack 3D Bins with a Bounded Buffer
A new method uses global item-space matching and geometry-aware ranking to improve irreversible packing decisions in buffered online 3D bin packing.
Can Self-Evolving Agents Keep Learning Without Forgetting?
A new arXiv paper examines whether large language model agents that evolve on their own can adapt to new tasks without losing prior knowledge.
OpenSaC: Bringing 'Search as Code' to Deep Research Agents
A new Python library treats search as a programmable component, potentially making deep research agents more modular and reproducible.
When Retrieval Conflicts with Skills: Procedural Hallucination
New research examines how language models hallucinate when retrieved information partially contradicts the procedures they have learned as skills.
Three-Stage Audit Separates Credential Exposure from Harm
A new arXiv paper proposes a three-stage audit to determine when an LLM agent's shared memory exposes one user's credentials to another and whether that exposure leads to actual misuse.
Correctness-filtered self-distillation: do all rollouts help?
A new arXiv paper questions the assumption that every student-generated trajectory provides useful supervision in on-policy self-distillation, proposing a correctness-filtered variant.
RTL Repair Benchmarks: What Do They Actually Measure?
A new dataset and evaluation audit show that conventional RTL repair scores can be skewed by reset semantics and specification baselines.
CODA: A cohort- and drift-aware model for clinical reasoning
A new arXiv preprint proposes a multimodal foundation model that accounts for patient cohorts and temporal drift, addressing a common blind spot in clinical AI.
Medical AI Fine-Tuning Shifts Risks, Says New Evaluation Framework
A new arXiv paper argues that evaluating patient-facing medical LLMs on factual correctness alone misses how fine-tuning changes their risk profile.
New metric isolates LLM contribution in biological screens
A proposed metric, CADS, separates an LLM proposer's incremental value from the discovery capacity of the screening harness.
Decision-Model Controllers Raise Agent Costs Without Boosting Success
A new arXiv study finds that delegating small decisions to a typed decision model increases agent costs without improving task success.
Selm proposes self-evolving library management for LLM agents
A new arXiv paper introduces Selm, a method for systematically managing evolving skill libraries in LLM agents.
TESS: A Four-Stage LLM Framework for Embodied Task Planning
A new arXiv paper proposes TESS, an LLM-guided method that decomposes complex embodied tasks into atomic actions through translation, exploration, simulation, and search.
SearCl: Automating Conversational Recommendation Dataset Creation
A new method generates conversational recommendation dialogues from user search interactions and knowledge graphs, reducing the need for manual annotation.
System X Lets LLM Decision Models Reply in Words from Their Own KV Cache
A new arXiv paper describes System X, an LLM decision model that augments probability outputs with a text-reply option drawn from its own KV cache.