Model Casting Sparsifies Feed-Forward Network Activations
A new mid-training recipe could make feed-forward layers in neural networks far more sparsely activated at inference time.
Research Digest
Preprints and papers, rewritten in plain English.
SAMBAR aims to help vision-language-action models learn new tasks without forgetting
A new arXiv preprint proposes a method to balance acquiring new manipulation skills and retaining previously learned ones in vision-language-action models.
New Matching Method Targets Molecular Inverse Design
A proposed graph-based distribution matching approach could improve multi-property control while preserving chemical validity.
TRIAGE: A New Framework for Evaluating LLM Unlearning
A proposed evaluation method, TRIAGE, aims to better assess how well large language models remove targeted knowledge without losing other capabilities.
CAFE: Fast Posterior Estimation for Counterfactual Prediction
A new arXiv paper proposes a method to make counterfactual prediction tractable despite the fundamental identifiability problem in observational data.
Guarded Freezing: A New Approach to Fine-Tuning Pretrained Models
A new method accounts for how updates to unfrozen layers alter inputs to frozen ones, preserving performance during fine-tuning.
NeuralCert Aims to Certify Neural Math Discoveries
A new framework proposes adding formal certification to neural-network results in mathematics, addressing their lack of exactness.
Staged Depth Training Gives PINNs a Representation Curriculum
A new arXiv preprint proposes an ordered representation curriculum for physics-informed neural networks to improve performance.
Point-in-Time Adaptation Tackles Look-Ahead Bias in Financial Models
A new method adapts financial language models to use only information available at the time of prediction, reducing look-ahead bias in backtests.
Strategic Self-Consistency: A New Twist on Majority Voting for LLMs
A new arXiv paper proposes a strategic approach to self-consistency, aiming to improve how large language models select answers from multiple reasoning paths.
Graph Networks Infer Hidden Forces from Coarse Motion Data
A new arXiv study proposes graph networks to recover unobserved mechanical forces from discretely sampled trajectories.
GeoAI Approach for EV Charging Station Placement
A new arXiv study applies geospatial artificial intelligence to select EV charging locations, shifting focus from cost minimization to broader siting criteria.
RAZOR prunes replaceable experts in MoE language models
A new pruning method targets replaceable experts in mixture-of-experts LLMs to cut storage while preserving output behaviour.
Q-Targets Make In-Context Reinforcement Learning Provably Robust to Weak Data
A new arXiv paper argues that training Transformers with Q-targets instead of behavior cloning gives in-context reinforcement learning stronger theoretical guarantees and less dependence on data quality.
Cost-Aware Dueling Bandits for Choosing the Best LLM
Researchers frame LLM selection as a multi-armed bandit problem that accounts for differing query costs, not just accuracy.
NestedTensor Aims to Eliminate Padding Waste in Deep Learning
A new composable tensor design targets the computational cost of dense batching for variable-size inputs.
LLM Agents Put Recommender Systems in Users' Hands
A new arXiv paper proposes shifting recommender systems from platform control to user side using LLM agents.
Activation Steering Lets LLMs Juggle Multiple Social Norms
A new paper proposes a lightweight causal steering method to help large language models adapt to interacting values without retraining.