Table Structure Extraction: Merged Cells

The short version: table parsing is a structure-recognition problem, not just text recognition. Only by restoring merged cells (rowspan / colspan) and the correct reading order can downstream systems treat a table as one complete object to retrieve.

1. Why tables are hard

A table in a document is not plain text; it carries row-column relations. Naive OCR recognizes each cell as text but loses "which cells merge into one." Once structure is gone, the table becomes scattered fragments in your data—either unfindable or wrongly assembled at retrieval.

Financial reports, research notes, and prospectuses are exactly the table-densest documents. Shred a cross-page income statement and a query for net profit hits a header fragment while a query for revenue hits a data row on another page; the assembled answer reads fluently and is wrong. In evaluation this shows up as retrieval hits with generation errors—the hardest class to debug, because every fragment looks correct on its own.

There are two common detours: hand the table screenshot to a general multimodal model, or force the whole table into CSV. The first does not scale and resists validation; the second misaligns the moment merged cells appear, leaving the hierarchy to guesswork. A table carries its own layout semantics, so the right move is to recognize structure at parse time instead of pushing the problem downstream—every rework in chunking and retrieval pays for what this step skipped.

2. TSR restores structure

Table Structure Recognition (TSR) is done by tsr.onnx. After layout analysis locates the table region, TSR emits HTML with rowspan / colspan, so merged cells are restored correctly rather than brutally split into tiny cells. The mapping between row headers, column headers, and data areas is pinned down with it.

Why emit HTML rather than plain text? Because row and column tags are semantics: span relations are recorded explicitly, downstream chunking can organize text row by row, and the LLM can read the table as a table. Restoring structure from fragments after the fact costs far more than keeping it at parse time.

3. Reading order decides retrieval correctness

Reading order answers "which cell first, which next." For cross-page tables, vertical headers, and nested tables, a wrong order produces semantically broken content. Local inference preserves reading order while restoring structure, so downstream chunks get human-readable table text.

Coordinates are preserved as well, and they power layout-level traceability: when a retrieved answer cites a number, you can trace it back to the exact page and region of the source document. For finance and audit workflows that require an evidence trail, this is not optional—it is what makes an answer believable.

4. How table metrics enter the knowledge base

In research scenarios, metrics inside tables (revenue, net profit) are extracted into a SQLite metric store via Unstructured-to-SQL, so they are both vector-retrievable and structurally queryable. The two channels complement each other: vector retrieval handles fuzzy semantic questions, SQL handles exact numeric ones.

The payoff is measurable. In one anonymized research corpus project, extracted table metrics served alongside the vector store, and retrieval smoke tests on 000001 and 000002 both hit 10/10—exact lookups no longer depended on dense-retrieval luck. Table structure rate likewise became one of the key corpus-level metrics.

Table structure recognition is part of layout analysis. For the full scan pipeline see Scanned PDF OCR and Layout Analysis; for turning recognition into chunks see Chunking Strategies and Financial Research RAG.

Real-World Scenario: From Table Recovery to a Metric Store

The endpoint of table recognition is not a table that looks right, but structured data you can query. In the A-share research RAG project, financial tables recovered by layout analysis feed a SQLite metric store (Unstructured-to-SQL); the two-issuer corpus alone yielded 457 metrics. When a user asks for a specific revenue figure of a specific fiscal year, the system can hit the metric record directly instead of letting the model guess from an OCR paragraph.

The critical prerequisite is merged-cell recovery: cross-row and cross-column headers are everywhere in research tables. When rowspan / colspan recovery fails, extracted metrics land on the wrong period or the wrong line item — and wrong data is more dangerous than missing data, because it looks true. How recovered tables pair with parent-chunk context is covered in Parent-Child Chunks; the full financial research case is in Financial Research RAG in Practice.

FAQ

Q1: How is table parsing different from text recognition?

Text recognition only turns cells into text; table parsing must also restore row-column relations and merged cells. Without structure the table is fragments—retrieval either misses it or assembles the wrong rows into a fluent-looking but wrong answer.

Q2: What restores merged cells?

tsr.onnx performs table structure recognition and emits HTML with rowspan / colspan, restoring merged cells correctly so downstream chunks can organize semantics row by row instead of splitting them into small cells.

Q3: What happens if reading order is wrong?

For cross-page, vertical-header, or nested tables, a wrong order yields semantically broken content, downstream chunks receive misordered text, and cited numbers can no longer be traced back to their source position.

Q4: How are table metrics made retrievable?

In research scenarios, table metrics are extracted via Unstructured-to-SQL into a SQLite metric store, enabling both vector retrieval and structured queries—exact lookups no longer depend on dense-retrieval luck.