LoRA Fine-Tuning on Phone NPUs: First-Order Costs Remain
A new arXiv paper describes on-device LLM fine-tuning with LoRA, noting that first-order training steps still carry a cost even after parameter reduction.
Research Digest
Preprints and papers, rewritten in plain English.
Auxiliary Observations Could Sharpen Bayesian Optimization
A new arXiv preprint proposes using paired auxiliary observations to reduce the impact of noise in Bayesian optimization.
Automating the Search for Pretraining Code Curriculums
A new method, PRECOCIAL, uses iterative search to discover code-based pretraining curriculums instead of relying on hand-designed programs.
Certified Sparse Attention: Perturbing Retrieval to Control Errors
A new arXiv paper proposes a perturbation method that gives sparse attention provable per-query error control, addressing heavy-tailed errors on real KV caches.
Uncertainty-Aware Token Merging Speeds Up Weather Models
A new method, UtoMe, guides token merging in weather transformers by accounting for the varying reliability of reanalysis data.
Structure of Shared Parameters Matters in Recursive Transformers
A new arXiv paper argues that how shared parameters are organized across depth is as important as how many are shared.
Arctic-SWE Tackles Reproducibility in Long-Horizon RL Agents
A new arXiv paper argues that reproducible RL baselines for long-horizon software engineering agents require co-designing the model and system.
RAFT: Filtering Redundant Inputs for Faster Temporal Graph Training
A new method called RAFT aims to reduce redundant data in temporal graph neural network training, potentially improving efficiency.
Impossibility Result for ETF Targets, Equinorm Remedy for ACL
A new proof shows a fundamental limit of equiangular tight frame targets in analytic continual learning, with an equinorm correction for imbalanced classes.
UpLoop upcycles pretrained LLMs into looped models without costly training
A new method, UpLoop, converts existing pretrained language models into looped architectures via a gated loop, avoiding the expense of training looped models from scratch.
Conjugate Compilation Improves Speculative Decoding for LLM Serving
A new arXiv paper proposes a self-improving co-compilation method to unlock stranded GPU capacity in large language model serving.
Synergy-Aware Flow Matching Improves Single-Cell Perturbation Prediction
A new method accounts for synergistic effects between perturbations to better predict single-cell responses.
Multifidelity Monte Carlo Extended to High-Dimensional Quantities of Interest
A new arXiv preprint extends multifidelity Monte Carlo estimation to high-dimensional quantities of interest, potentially reducing cost when high-fidelity models are expensive.
Latent world models should plan along the manifold, not the Euclidean shortcut
A new paper argues that latent world models make a basic geometric mistake by measuring goal distances with Euclidean norms, and should instead use distances along the latent manifold.
CoEvo-RSI: Co-evolving Models and Harnesses for Self-Improving Agents
A new arXiv preprint proposes a framework where recursive self-improving agents jointly evolve their execution scaffolds and intrinsic capabilities.
Distillation overshoot: weaker students may outperform stronger teachers
New research identifies and characterises a counterintuitive 'overshoot' effect in agentic on-policy distillation, where smaller students surpass their teachers.
PACT: Cutting LoRA Fine-Tuning Costs on Consumer GPUs
A new arXiv preprint proposes a data-flow LoRA fine-tuning approach aimed at the compute and memory costs that remain on consumer GPUs.
Decentralized online convex optimization tackles exp-concave losses
A new arXiv paper extends the decentralized online convex optimization framework to exp-concave loss functions, though the abstract's results are cut off in the available text.