Extreme weight quantization—below four bits—offers clear gains in memory and bandwidth, but the trade-off is often hidden in task performance. According to the paper's abstract, perplexity may look acceptable while factual knowledge and task accuracy degrade. This is the problem the authors of RoleQ set out to address.
RoleQ is a method for converting language models to ternary weights, meaning weights are restricted to three values. The abstract notes that ternary weights have mostly been obtained by training from... (the text is cut off), but RoleQ appears to take a different route, using role-aligned scales to preserve what standard quantization loses.
The paper is available on arXiv under reference 2610.21808. As the abstract is truncated, the full details of the role-aligned scaling mechanism and evaluation results are not yet visible in this summary.