RAG Chunking Strategies: Fixed to Type-Aware

The short version: chunking is not one "how long" parameter but "by what structure." Fixed-size splitting causes semantic breaks; type-aware presets balance recall against context completeness.

1. The fixed-size trap

Fixed-size splitting is the easiest pipeline to build: set a character count, slice from the top, keep a little overlap, and ship. It is also the top source of semantic breaks. A definition can land exactly on a cut, and the first half retrieved alone is incomplete, its embedding distorted. When tables mix with prose, equal-length cuts shred row-column relationships, and a number gets separated from the metric it belongs to.

The damage concentrates on strongly structured content: technical docs with numbered sections, contracts with nested clauses, Q&A pairs with a natural boundary between question and answer. Equal-length slicing flattens all of it. Practitioners widely observe that roughly 80% of RAG quality problems trace back to data handling, and chunking is the most underestimated part of that handling. Fixed-size splitting suits structureless plain text; it should not be the default for an enterprise corpus.

2. Structure-aware: chapter tree to parent-child

The steadier path follows the document itself: parse the chapter tree, atomicize paragraphs, bin-pack under size constraints, then organize into parent-child chunks. Order matters. Respect the structure first, then apply size constraints inside it, rather than forcing character counts onto the structure. Authors break documents into sections for a reason, and chunking should follow that intent.

Two guarantees follow. Each chunk stays under the model context window, because oversized passages are split further during packing. And semantic boundaries align with writing boundaries: what a section discusses is what the chunk contains, so a hit is relevant and a citation is complete. Preprocessing consumes 40%–50% of enterprise RAG engineering effort, and getting chunking right is one of the highest-leverage investments in that pipeline.

3. Four type-aware presets

Different document types react differently to size and overlap; a single parameter set cannot serve them all. We ship four presets (chunk_size / overlap):

The logic transfers directly. Technical docs have short sections and dense terminology, so 800/150 keeps chunks tight instead of mixing three topics into one. Legal clauses interlock, so 1500/300 preserves context and prevents out-of-context citations. Q&A pairs are naturally standalone at 500/100. When unsure, start from the general 1000/200 and tune against evaluation data.

4. Chunking is not the finish line

A bare chunk is only a passage, and the signals it offers retrieval are limited. After splitting, each chunk should get four-level enhancement: breadcrumb (section path), keywords, summary, and a HyDE hypothetical question, so retrieval hits precisely while still understanding context. Details in Chunk Enrichment.

The operational risk shows up in bulk ingests. During earnings season, hundreds of announcements arrive in one batch; if the chunking parameters are wrong, every downstream retrieval and generation step has to be redone. Smoke-test on a small corpus first and confirm before scaling—it is the cheapest insurance in the whole pipeline.

Chunking follows desensitization and compression. The upstream steps are Document Cleaning and Desensitization Flow; for the parent-child structure after splitting, see Parent-Child Chunks.

Engineering Checklist: Choosing Among the Four Presets

Chunking parameters should not be guessed; start from presets matched to document type and tune from there:

After choosing, verify: re-run evaluation to confirm HitRate@K and MRR did not regress with the new chunking — do not assume finer equals better. How parent-child structure balances fine-grained retrieval and full context is covered in Parent-Child Chunks; gating chunk changes with regression tests in RAG Regression Gate.

FAQ

Q1: Why is fixed-size splitting bad?

It hard-cuts by character count, so a definition or paragraph often lands on a cut and the retrieved half is semantically incomplete. It is the top source of semantic breaks and hurts structured content most—technical docs, contracts, and Q&A pairs all get their boundaries flattened. Prefer structure-aware splitting with type-based presets.

Q2: What are the four presets for?

General 1000/200, Technical 800/150, Legal 1500/300, and Q&A 500/100. Technical docs are shorter and denser, legal docs need longer chunks to keep interlocking clauses in context, and Q&A pairs are standalone. When unsure, start from the general preset and tune against evaluation results.

Q3: What is the structure-aware flow?

Follow the document chapter tree → atomic units → bin-packing → parent-child chunks. Respect the author's structure first, then apply size constraints inside it, so each chunk stays under the model context window while semantic boundaries stay aligned with sections.

Q4: What comes after splitting?

Each chunk gets four-level enhancement: breadcrumb (section path), keywords, summary, and a HyDE hypothetical question, so retrieval hits precisely while keeping context. Also smoke-test on a small corpus before a bulk ingest, so a wrong parameter does not force a full redo downstream.