Scanned PDF OCR and Layout Analysis

The short version: scanned PDF parsing is a two-stage local-inference job—layout analysis then OCR—with no cloud dependency. In a private deployment, both the OCR and layout models run on a local ONNX runtime, so data never leaves your perimeter.

1. Why scans are hard

A scan is essentially a stack of images with no selectable text layer. Feed it to a text-only parser and you get empty or garbled output. Stamped contracts, expense receipts, historical archives, and government red-header documents arrive as scans in volume, so this class of data cannot be avoided—only handled properly.

The right shape is two-stage: layout analysis first to locate text blocks, then OCR to turn images into text. Skip either half and the result breaks—OCR without layout gives you text in the wrong order and hierarchy, layout without OCR gives you a pile of empty boxes with coordinates and nothing inside. Run them in sequence and a scan becomes retrievable text.

A common detour is feeding whole-page images to a general multimodal model. It works in a demo and collapses in production: page counts drive time and cost up linearly, and the output resists structured validation, so you cannot tell whether the parse is right. The two-stage layout-plus-OCR route leaves an inspectable intermediate artifact at every step, so failures localize to a specific stage instead of a final text you have to squint at.

2. Two-model local inference

DeepDoc layout analysis runs layout.onnx and tsr.onnx as local ONNX inference: the first locates regions like titles, paragraphs, tables, and images; the second does table structure recognition (TSR). OCR uses PP-OCRv6 and PDF rendering uses PDFium. No step in the chain calls an external API, which satisfies the hard data-sovereignty requirement in finance and government.

Local inference is not only a compliance story; it is also about cost and stability. Pushing tens of thousands of scanned pages through a per-page cloud OCR API compounds fees and ties data availability to someone else's uptime. On a local ONNX runtime, throughput scales with the machine, the pipeline keeps running offline, and private-delivery security reviews get one item shorter.

3. Preserve three things

The classic failure is "characters recognized but structure lost." A local pipeline must preserve reading order (which paragraph precedes which), coordinates (where each block sits on the page), and table HTML structure (including rowspan / colspan merged-cell restoration). With all three, downstream chunking understands layout semantics.

One more stage follows: scans almost always carry headers, footers, page numbers, and watermarks, and the cleaning stage detects and strips them automatically, emitting a cleaning report for the record. Sensitive content such as seals or handwritten annotations must be desensitized before chunking and embedding—skip that and there is no way to patch it later.

4. Quantify parse quality

Scan parsing should not be judged by feel. We score structure, relation, and content with parse_quality and assign an overall grade. In one measurement, a 443-page OFD document completed the full parse pipeline in about 124 seconds and scored 0.924 on the three-layer quality check (structure layer perfect). The document mixes tables, multi-column text, and images—a typically hard sample for a scan batch, and quantified output like this is what feeds the evaluation loop.

A low score does not mean rerunning the batch; it means triage. Low-scoring documents route to human review or a parameter retry, high-scoring ones flow straight into the knowledge base. Once those numbers sit inside the evaluation loop, every model swap and threshold change gets a before-and-after comparison, and tuning stops being guesswork.

Scans are one slice of parsing. For differential handling beyond scans see Parsing Legacy Document Formats; to turn recognition output into a retrievable knowledge base, see Document Cleaning and Chunking Strategies.

Engineering Checklist: OCR Routing and Three-Layer QC

• Route OCR, do not enable it globally: text-native PDFs extract in under 200ms each; only scanned pages are routed to OCR automatically. Global OCR inflates parsing time for clean PDFs by an order of magnitude and turns rendering noise into phantom characters.
• Run layout models locally: the layout.onnx + tsr.onnx pair runs as local ONNX inference, preserving reading order, coordinates, and table HTML structure including rowspan / colspan — a hard requirement for compliance-bound scanned documents.
• Pin the OCR engine version: recognition precision and recall differ noticeably across PP-OCR versions; pinning the version is what makes parsing quality reproducible and regression-testable.
• Use the three-layer QC score as an acceptance gate: a 443-page OFD document completes the full parsing flow in about 124 seconds, scoring 0.924 on the three-layer quality check with a perfect structure score. Without that score you cannot tell whether a batch of scans is fit for ingestion.

When QC scores drop, suspect layout analysis before the OCR engine itself — most misreads are actually reading-order errors. For scanned tables see Table Structure Extraction; for mixed legacy corpora see Legacy Document Formats.

FAQ

Q1: Why isn't OCR alone enough for scans?

OCR only turns images into text; it does not understand which block is a title or which cells belong to one table. Layout analysis must locate regions first, otherwise order and structure break and chunks arrive as fragments.

Q2: How is local ONNX inference different from cloud OCR?

Every model (layout.onnx, tsr.onnx, PP-OCRv6) runs on your own machine with no external API calls, so data never leaves your perimeter—meeting finance and government compliance—while avoiding per-page cloud fees.

Q3: How are merged table cells restored?

tsr.onnx performs table structure recognition and emits HTML with rowspan / colspan, so merged cells are restored correctly instead of being split into scattered small cells that break row semantics.

Q4: How is scan parse quality measured?

parse_quality scores structure, relation, and content and gives an overall grade. In measurement, a 443-page OFD document finished in about 124 seconds at 0.924 quality (structure layer perfect); low-scoring documents route to review automatically.