This batch of preprints splits into two camps. Several papers attack the cost of LLM inference and training. SpecQuant combines speculative decoding with multi-parent quantization for adaptive inference on consumer hardware. Another quantization paper uses activation-guided compensation and orthogonal residuals to make aggressive W4A4 quantization more tractable. VQ-Logits compresses the output vocabulary bottleneck, and an L0-regularized mixture-of-experts method accelerates dense LLMs. Souper-Model claims simple arithmetic can unlock state-of-the-art performance during training. These approaches differ in technique but share the goal of reducing resource demands.
A second group examines reliability. One paper argues LLMs are Shannon lossy compressors rather than Solomonoff induction estimators, suggesting fundamental limits without symbolic model synthesis. Another finds LLM agents exhibit "constraint decay" when generating backend code under strict structural requirements. A survey of complex problem solving identifies early-step error amplification, prompt brittleness, and failure to revise. Small language models under 3 billion parameters can use entropy-based confidence signals to improve accuracy, according to another study. A hierarchical memory architecture is proposed to overcome context limits in long-horizon multi-agent workflows. Finally, one paper studies efficient recurring benchmarking for a production analytics agent, since full benchmarks are costly to rerun.
Where the papers agree, it is on the broad diagnosis: LLMs are powerful but expensive and brittle. They differ sharply on remedies. Efficiency papers propose architectural and algorithmic changes, while reliability papers offer diagnostic frameworks, memory systems, confidence calibration, or cheaper evaluation. Notably, the theoretical paper warns that scaling alone will not reach general intelligence, a claim that sits in tension with the practical engineering optimism of the acceleration papers.