The short version: roughly 80% of RAG quality problems are not model problems, they are data problems. RAG knowledge quality equals model capability times data quality, and data quality is the part that usually breaks first.
1. Why data is the first suspect
In enterprise RAG, document preprocessing (parse, clean, desensitize, chunk, annotate) accounts for 40%–50% of the engineering effort, yet it is the step teams most often rush. Everyone enjoys tuning prompts and swapping models; few go back to check what each ingested document actually became. No matter how strong the model, if you feed it broken tables, scrambled layouts, or truncated paragraphs, the answers will be wrong. Poor document parsing is the single largest cause of RAG hallucinations, especially in document-heavy domains like finance, law, and government.
Teams often blame the model first. But the same model on clean data versus dirty data can differ by orders of magnitude. Worse, data problems rarely show up early: a demo built on dozens of curated documents looks flawless, then quarterly report season floods the pipeline with hundreds of scans and filings, and retrieval starts assembling the wrong table rows. Data quality is not a nice-to-have; it is the foundation everything else stands on.
2. Four recurring data failure modes
Across real corpora we keep seeing the same four problems, each mapping to an engineering blind spot. Put the symptom next to the root cause and audit your own system against it:
| Failure | Symptom | Root cause |
|---|---|---|
| Semantic cut | A passage split in half, only half retrieved | Fixed-size chunking, no structure |
| Missed recall | Model numbers, IDs, metric terms not found | Dense-only retrieval, no BM25 |
| Leakage | Junior staff sees confidential sections | No permission control or post-filter |
| Stale citations | Outdated policies still cited | No version or validity management |
What these four have in common is timing: they stay quiet in demos and erupt at scale. When a nationwide commercial bank ingested 859 historical disclosure documents in batch, layered parse-quality scoring and retrieval smoke tests (000001 and 000002 both hitting 10/10) were what confirmed the pipeline had silently dropped nothing and leaked nothing. Teams without such checks usually learn about semantic breaks and stale citations from user complaints.
3. Order matters more than tooling
The correct pipeline order is: parse → clean → desensitize → compress → chunk → govern → evaluate. Get the order wrong and everything downstream is patching holes. Desensitization must happen before chunking and embedding, because the embedding itself is a leakage surface—sensitive fields baked into vectors are secrets written into your database, and no amount of post-filtering removes them.
Compression belongs after desensitization too: prefer the desensitized copy as the compression source, average roughly 60% storage savings, and gate over-compression with SSIM / PSNR so the layout does not distort. Chunking has its own order—rebuild the section tree with layout analysis first, then atomize and pack; fixed-size chunking only manufactures semantic breaks. Every step feeds the next one, and time saved by skipping steps gets repaid with interest.
4. Close the loop with evaluation
Turn "is the data good" into a number you can track. We recommend three metric layers: corpus-level (parse completeness, table structure rate, OCR accuracy, desensitization coverage, permission hit rate), retrieval-level (HitRate@K, Recall@K, MRR, NDCG@K), and generation-level (Faithfulness, AnswerRelevancy, ContextPrecision). With that ruler in place, any change to a data rule is verified automatically by a regression gate instead of waiting for a thumbs-down.
The loop also makes optimization evidence-based. Chunk parameters, cleaning rules, parser upgrades—each can run as a controlled experiment on a small set before rolling out, and with SHA-256 content hashing for idempotency plus incremental diffs, only changed chunks get re-embedded, keeping iteration cost bounded.
Not sure how far your corpus is from production-ready? Start with our RAG data engineering overview, or jump to the 30-item RAG launch checklist. For real-scale numbers, see the anonymized customer case.
Real-World Scenario: A Nationwide Commercial Bank
Data quality sounds abstract until you look at production numbers. In a knowledge base project covering two listed issuers — a nationwide commercial bank and a property developer — 859 documents were processed with zero failures, producing 36,395 chunks and roughly 35.19 million characters across 25 reporting years (2001-06 to 2026-08). The document mix is representative: 673 PDFs, 112 HTML files, 74 DOCX files, plus 227 research reports from 25 institutions. Zero failures came from data engineering, not from the model: corpus-level metrics such as parsing completeness, desensitization coverage, and permission hit rate were all verified before ingestion. In retrieval smoke tests, both tickers 000001 and 000002 scored 10/10 hits, and ten end-to-end questions on 000002 completed in 4.6 seconds.
To assess your own knowledge base, run a three-layer self-check: corpus layer for parsing completeness and table structure rate, retrieval layer for HitRate@K and MRR, generation layer for Faithfulness. Metric definitions are covered in RAG Evaluation Metrics, and a full pre-launch checklist in the 30-item RAG launch checklist.
FAQ
Q1: RAG is underperforming—bigger model or better data first?
Check data first. Around 80% of failures trace back to poor preprocessing; no model recovers from broken layouts, scrambled tables, or leaked fields. Build corpus-, retrieval-, and generation-level metrics to locate the problem with numbers before touching the model.
Q2: How much of RAG engineering is document preprocessing?
About 40%–50% of enterprise RAG effort. It covers parsing, cleaning, desensitization, chunking, and metadata governance—the largest and most underestimated slice of the work. If this layer is weak, every later retrieval and generation fix is just patching holes.
Q3: Why must desensitization happen before embedding?
Because the embedding is itself a leakage surface. Baking sensitive fields into vectors writes the secret into your vector database; post-hoc filtering cannot undo an embedding that already leaked. Desensitize first, then chunk and embed.
Q4: How do I know if my corpus data quality is good enough?
Run three metric layers on a schedule: corpus-level parse completeness and desensitization coverage, retrieval-level HitRate@K and MRR, generation-level Faithfulness and AnswerRelevancy. Ship only when the scores clear the bar; the launch checklist walks through each item.
Related Articles
Need Production-Ready RAG Data Pipelines? Meet UGLYPEAR AI
A privately deployed RAG data engineering platform: parsing, cleaning, redaction, compression, chunking, governance, and evaluation in one on-premises pipeline — fully local inference, your data never leaves your domain.