Legal documents are a hard case for retrieval-augmented generation. Their arguments are organised hierarchically, and standard chunking tends to break that structure apart. The result is that retrieved passages lose the context needed to ground an LLM's answer.
A new arXiv paper, StrucChunk, takes aim at this problem. The authors argue that generic embeddings also struggle with precision in this domain, though the abstract is cut off before the full description of the method. The title and framing suggest a structure-aware retrieval approach that respects the document's organisation.
If it works as described, such an approach could make RAG more reliable for legal research and analysis, where preserving argument structure matters as much as finding the right passage.