Large language models and vision-language models are increasingly deployed on a wide range of hardware, from powerful data-center GPUs to edge devices with limited resources. However, their fixed-size architectures make it difficult to reuse a single model across such varied environments, according to a new arXiv paper.
The paper introduces HydraLLM, an approach based on adaptive width slicing. Instead of training separate models for each hardware target, HydraLLM aims to adjust the model's width dynamically, allowing one model to serve multiple deployment scenarios. The abstract notes that this applies to both text-only LLMs and multimodal vision-language models.
As described, the work directly addresses the mismatch between static model sizes and the diversity of real-world hardware. The abstract does not provide implementation details or experimental results, so the reported contribution is the conceptual framework for elastic model reuse.