- Updated: August 26, 2026
- 2 min read
Semantic Compression Trees: Multi‑Resolution Knowledge Retrieval via Hierarchical Semantic Residuals
Semantic Compression Trees: Multi‑Resolution Knowledge Retrieval via Hierarchical Semantic Residuals
Published on ubos.tech
Retrieval‑augmented generation (RAG) traditionally relies on flat, fixed‑granularity indexes where documents are split into uniform chunks and retrieved by similarity. This approach discards the inherent hierarchical structure of source material, leading to inefficiencies at scale.
In this article we present Semantic Compression Trees (SCT), a novel hierarchical index that stores only the semantic residual at each node—the incremental information beyond its parent. Retrieval proceeds via progressive descent from the root, making the per‑query cost a function of tree depth rather than collection size.
Key findings from our evaluation on the QASPER benchmark (50 papers, 173 questions) include:
- When the relevant document is supplied, SCT with a zero‑LLM extractive compressor matches dense retrieval on answer quality (0.274 vs. 0.277 F1, p = 0.37) while using 30 % fewer context tokens and no LLM calls to build the index.
- Residual storage outperforms storing full summaries at each node (0.274 vs. 0.205, p < 0.001).
- Scaling the collection fifty‑fold multiplies flat retrieval’s per‑query scoring work by 48.9×, whereas SCT’s cost grows only 6.4×.
- Progressive descent itself is not yet supported; retrieving the same residuals without the tree yields identical performance when the document is given, but performance degrades when the system must select the document (0.122 vs. 0.165 F1, p < 0.001).
- Routing accuracy is a limiting factor: descent selects the correct paper 20.2 % of the time versus 39.3 % for flat retrieval, because routing decisions are made from the highly compressed root residual.
These results suggest that while the residual representation is valuable, top‑down routing requires further research.
For a visual overview of the hierarchical structure, see the illustration below:
Read more about our research and related projects on the Ubos Tech research page.
— End of article —
Andrii Bidochko
CTO UBOS
Andrii Bidochko is an AI entrepreneur and researcher focused on AI agents, reinforcement learning, and autonomous systems. He writes about the technologies shaping the future of machine intelligence, from frontier models and agent architectures to real-world AI applications.