Hierarchy Aware Chunker
Hierarchy Chunker for RAG | No Overlaps, No Tweaking Needed
4 followers
Hierarchy Chunker for RAG | No Overlaps, No Tweaking Needed
4 followers
Hierarchy-Aware Document Chunker — The Next Generation of Document Chunking | Preserve context, structure, and meaning with layout aware chunking — No more chunk overlaps and spending hours tweaking chunk sizes.
Introducing Hierarchy-Aware Document Chunker — no more broken context across chunks 🚀
One of the hardest parts of RAG is chunking:
Most standard chunkers (like RecursiveTextSplitter, fixed-length splitters, etc.) just split based on character count or tokens. You end up spending hours tweaking chunk sizes and overlaps, hoping to find a suitable solution. But no matter what you try, they still cut blindly through headings, sections, or paragraphs ... causing chunks to lose both context and continuity with the surrounding text.
So I built a Hierarchy Aware Document Chunker.
✨Features:
- 📑 Understands document structure (titles, headings, subheadings, sections).
- 🔗 Merges nested subheadings into the right chunk so context flows properly.
- 🧩 Preserves multiple levels of hierarchy (e.g., Title → Subtitle→ Section → Subsections).
- 🏷️ Adds metadata to each chunk (so every chunk knows which section it belongs to).
- ✅ Produces chunks that are context-aware, structured, and retriever-friendly.
- Ideal for legal docs, research papers, contracts, etc.
- Works great for Multi-Level Nesting.
No preprocessing needed — just paste your raw content or Markdown and you’re are good to go !
Flexible Switching: Seamlessly integrates with any LangChain-compatible Providers (e.g., OpenAI, Anthropic, Google ).