Why it matters
Chunking, embedding and citation all assume the text is in reading order and that a heading belongs to the section beneath it. Extractors that emit text in storage order violate that assumption: a two-column page interleaves, a table becomes a stream of cells, a footnote lands mid-sentence. The damage is invisible in a spot check of prose and decisive when the answer sits in a table.
Mental model
Parsing is transcription from a photograph of a page: you are reconstructing what a human would read, not what the file format stores. The metaphor breaks because a human reading a page uses meaning to resolve order, while a parser must decide order from geometry before it knows what the text says.
How it works
Synthetic research brief · page 3 of 12
Two columns, one table, one footnote
bounding boxes
Step 1Coordinates, not paragraphs
The text layer gives positioned glyphs. Nothing in it says which line is a heading, which block is a table, or which column to read next.
ACME RESEARCH · QUARTERLY BRIEF · 2026 Q3
Retention after the plan change
What changed
Who stayed
| Cohort | Renewed | Churn |
|---|---|---|
| Annual | 91% | 4% |
| Monthly | 72% | 19% |
| Returning | 64% | 27% |
| New | 88% | 7% |
Table 1. Renewal and churn by cohort, first renewal after the change.
ACME RESEARCH · CONFIDENTIAL · PAGE 3
Step record
- File
- brief.pdf · 12 pages
- Text layer
- positioned glyphs
- Reading order
- not stored
The text version summarises this figure; select a region for its full text.
Figure. A two-column brief parsed twice: once in storage order, which interleaves the columns, and once by coordinates, which restores the column reading order and keeps the table as a table. Bounding boxes are the parser's only input; the heading hierarchy and the offsets are what it has to add.
What to notice. The page is synthetic and the block-level order stands in for the line-level content stream, so the interleaving is a shape, not a byte trace. Coordinates and character offsets are illustrative; the figure shows what structure recovery has to decide, not one library's output.
Text version
A synthetic two-column research brief. The left column holds a title, byline, two headings, three paragraphs and a method note; the right column holds a paragraph, a three-column table of cohorts with renewal and churn figures, its caption and a pull quote. In the storage-order step, dashed arrows zig-zag between the columns, so the first left-column paragraph is followed by the first right-column paragraph. In the reading-order step the arrows run down the left column first, then cross once to the right column. Bounding boxes mark every region, the table keeps its row and column grid, and a numbered badge marks each region in the order. Selecting a region shows its text; in the offsets step its page and character span also appears under the region.
- Detect the real format first, since extensions lie and scanned PDFs contain no text layer.
- Extract with a parser that exposes blocks, coordinates and font information rather than plain text only.
- Reconstruct reading order per page using column detection, then stitch pages in sequence.
- Detect tables as tables and serialise them with row and column structure, or convert them to key-value text deliberately.
- Attach heading hierarchy to the paragraphs beneath it so chunkers can split on section boundaries.
- Record page numbers and character offsets for every block so citations can resolve to a location.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Plain text extraction | Fast and simple | Loses tables, columns and headings | The corpus contains reports or forms |
| Layout-aware parser | Preserves structure and order | Slower, heavier dependencies | Throughput matters more than structure |
| Table serialisation to markdown | Readable by models and chunkers | Wide tables wrap and lose alignment | Tables have many columns or merged cells |
| Vision-model extraction | Handles complex layouts and scans | Cost, latency and nondeterminism | Documents are high-volume and simple |
Good for
Fast and simple
Cost
Loses tables, columns and headings
Fails when
The corpus contains reports or forms
- The corpus contains reports or forms
- Throughput matters more than structure
- Tables have many columns or merged cells
- Documents are high-volume and simple
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Plain text extraction is good for Fast and simple, costs Loses tables, columns and headings, and fails when The corpus contains reports or forms. Layout-aware parser is good for Preserves structure and order, costs Slower, heavier dependencies, and fails when Throughput matters more than structure. Table serialisation to markdown is good for Readable by models and chunkers, costs Wide tables wrap and lose alignment, and fails when Tables have many columns or merged cells. Vision-model extraction is good for Handles complex layouts and scans, costs Cost, latency and nondeterminism, and fails when Documents are high-volume and simple
Failure modes
- Column interleaving. Text from two columns alternates line by line, producing sentences that mix topics. Debug: print the extracted text of one two-column page and look for abrupt topic switches mid-paragraph.
- Table flattened to cell soup. Numbers lose their row and column association, so the model reads the wrong value. Debug: extract a known table and check that each row still contains its labels.
- Header and footer repetition. Running headers are indexed as content and pollute every chunk. Debug: count the most frequent lines across pages; repeated headers will top the list.
In production
Sample parsed output per source type on a schedule and diff it against a hand-checked reference. Track the share of documents with no extracted text, which usually means a scan without OCR.
Common misconceptions
People think: Text extraction and parsing are the same thing.
Actually: Extraction produces characters; parsing produces structure, order and boundaries that chunking depends on.
People think: PDF text is stored in reading order.
Actually: It is stored as positioned glyphs, and reading order must be inferred from coordinates.
Interview angle
Separate extraction from structure recovery, and name reading order, tables and headings as the things worth preserving. Tie parse quality to citation offsets. Saying you use a PDF library, without being able to explain why a two-column page produces garbled chunks, is the answer to avoid.
Sources
- Apache Tika — format detection and structured text extraction.
- PyMuPDF documentation — accessing blocks, coordinates and page structure from PDFs.
- Unstructured documentation — typed document elements such as titles, tables and list items.