Structured pruning reduces the inference cost of large language models by permanently removing neurons and weight rows, yielding dense, hardware-agnostic speedup. This makes it an attractive alternative to unstructured methods that require specialized hardware support.
Existing post-training methods for structured pruning couple neuron selection with weight reconstruction. SARATHI, described in a recent arXiv preprint, decouples these stages, using a data-free probing step and an adaptive reconstruction step.
The provided abstract text does not include experimental outcomes, so the practical impact of SARATHI is not yet detailed. The paper is listed on arXiv as 2610.15514v1.