Fine-tuning large language models on consumer GPUs is squeezed by both computation and memory. LoRA is a standard remedy because it shrinks the number of trainable parameters and the optimizer state, but it does not remove every cost: the frozen projection weights still demand compute, and activations kept for the backward pass consume memory.
The new preprint, PACT, proposes a data-flow approach to LoRA fine-tuning that targets these remaining bottlenecks. The title points to NVFP4 data-flow, suggesting a low-precision path through the frozen projections, though the abstract is cut off before the method is fully described.
Because there is only one source and its abstract is truncated, I cannot report experimental results or compare PACT with other approaches. The claims above are limited to what the abstract explicitly states.