Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 20 of 57

Parsing

0 of 57 complete

Why it matters

Chunking, embedding and citation all assume the text is in reading order and that a heading belongs to the section beneath it. Extractors that emit text in storage order violate that assumption: a two-column page interleaves, a table becomes a stream of cells, a footnote lands mid-sentence. The damage is invisible in a spot check of prose and decisive when the answer sits in a table.

Mental model

Parsing is transcription from a photograph of a page: you are reconstructing what a human would read, not what the file format stores. The metaphor breaks because a human reading a page uses meaning to resolve order, while a parser must decide order from geometry before it knows what the text says.

How it works

The page the parser receivesInteractive

Synthetic research brief · page 3 of 12

Two columns, one table, one footnote

bounding boxes

Step 1/6 · Coordinates, not paragraphs

Step 1Coordinates, not paragraphs

The text layer gives positioned glyphs. Nothing in it says which line is a heading, which block is a table, or which column to read next.

ACME RESEARCH · QUARTERLY BRIEF · 2026 Q3

, running header, region 1. Select to inspect.

Retention after the plan change

, title, region 2. Select to inspect.
Text

Field note · Growth analytics · 12 September 2026

, paragraph, region 3. Select to inspect.

What changed

, heading, region 4. Select to inspect.
Text

Annual plans moved to a single renewal date in March. Monthly plans kept their existing cycle, but the cancellation window was shortened from fourteen days to seven.

, paragraph, region 5. Select to inspect.

Who stayed

, heading, region 6. Select to inspect.
Text

Renewal held steady for annual plans and fell sharply for monthly plans in the first cohort after the change. The drop is concentrated in accounts that had cancelled once before and returned.

, paragraph, region 7. Select to inspect.
Text

Method: 4,218 accounts that renewed at least once. Percentages are rounded to the nearest point; churn counts voluntary cancellations only.

, paragraph, region 8. Select to inspect.
Text

The brief compares two cohorts on the same renewal date. Both cohorts were exposed to the new cancellation window; only the annual cohort saw a single renewal date.

, paragraph, region 9. Select to inspect.
TABLE 1
CohortRenewedChurn
Annual91%4%
Monthly72%19%
Returning64%27%
New88%7%
, table TABLE 1, region 10. Select to inspect.

Table 1. Renewal and churn by cohort, first renewal after the change.

, caption, region 11. Select to inspect.
Text

“The drop is concentrated in accounts that had cancelled once before and returned.”

, paragraph, region 12. Select to inspect.
Text

Source: synthetic cohort extract, 4,218 accounts. Percentages rounded to the nearest point.

, paragraph, region 13. Select to inspect.

ACME RESEARCH · CONFIDENTIAL · PAGE 3

, running footer, region 14. Select to inspect.
14 regions

Step record

File
brief.pdf · 12 pages
Text layer
positioned glyphs
Reading order
not stored

The text version summarises this figure; select a region for its full text.

Figure. A two-column brief parsed twice: once in storage order, which interleaves the columns, and once by coordinates, which restores the column reading order and keeps the table as a table. Bounding boxes are the parser's only input; the heading hierarchy and the offsets are what it has to add.

What to notice. The page is synthetic and the block-level order stands in for the line-level content stream, so the interleaving is a shape, not a byte trace. Coordinates and character offsets are illustrative; the figure shows what structure recovery has to decide, not one library's output.

Text version

A synthetic two-column research brief. The left column holds a title, byline, two headings, three paragraphs and a method note; the right column holds a paragraph, a three-column table of cohorts with renewal and churn figures, its caption and a pull quote. In the storage-order step, dashed arrows zig-zag between the columns, so the first left-column paragraph is followed by the first right-column paragraph. In the reading-order step the arrows run down the left column first, then cross once to the right column. Bounding boxes mark every region, the table keeps its row and column grid, and a numbered badge marks each region in the order. Selecting a region shows its text; in the offsets step its page and character span also appears under the region.

  1. Detect the real format first, since extensions lie and scanned PDFs contain no text layer.
  2. Extract with a parser that exposes blocks, coordinates and font information rather than plain text only.
  3. Reconstruct reading order per page using column detection, then stitch pages in sequence.
  4. Detect tables as tables and serialise them with row and column structure, or convert them to key-value text deliberately.
  5. Attach heading hierarchy to the paragraphs beneath it so chunkers can split on section boundaries.
  6. Record page numbers and character offsets for every block so citations can resolve to a location.

Trade-offs

ChoiceGood forCostFails when
Plain text extractionFast and simpleLoses tables, columns and headingsThe corpus contains reports or forms
Layout-aware parserPreserves structure and orderSlower, heavier dependenciesThroughput matters more than structure
Table serialisation to markdownReadable by models and chunkersWide tables wrap and lose alignmentTables have many columns or merged cells
Vision-model extractionHandles complex layouts and scansCost, latency and nondeterminismDocuments are high-volume and simple
Trade-off mapInteractive

Good for

Fast and simple

Cost

Loses tables, columns and headings

Fails when

The corpus contains reports or forms

  1. The corpus contains reports or forms
  2. Throughput matters more than structure
  3. Tables have many columns or merged cells
  4. Documents are high-volume and simple

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Plain text extraction is good for Fast and simple, costs Loses tables, columns and headings, and fails when The corpus contains reports or forms. Layout-aware parser is good for Preserves structure and order, costs Slower, heavier dependencies, and fails when Throughput matters more than structure. Table serialisation to markdown is good for Readable by models and chunkers, costs Wide tables wrap and lose alignment, and fails when Tables have many columns or merged cells. Vision-model extraction is good for Handles complex layouts and scans, costs Cost, latency and nondeterminism, and fails when Documents are high-volume and simple

Failure modes

  • Column interleaving. Text from two columns alternates line by line, producing sentences that mix topics. Debug: print the extracted text of one two-column page and look for abrupt topic switches mid-paragraph.
  • Table flattened to cell soup. Numbers lose their row and column association, so the model reads the wrong value. Debug: extract a known table and check that each row still contains its labels.
  • Header and footer repetition. Running headers are indexed as content and pollute every chunk. Debug: count the most frequent lines across pages; repeated headers will top the list.

In production

Sample parsed output per source type on a schedule and diff it against a hand-checked reference. Track the share of documents with no extracted text, which usually means a scan without OCR.

Common misconceptions

People think: Text extraction and parsing are the same thing.

Actually: Extraction produces characters; parsing produces structure, order and boundaries that chunking depends on.

People think: PDF text is stored in reading order.

Actually: It is stored as positioned glyphs, and reading order must be inferred from coordinates.

Interview angle

Separate extraction from structure recovery, and name reading order, tables and headings as the things worth preserving. Tie parse quality to citation offsets. Saying you use a PDF library, without being able to explain why a two-column page produces garbled chunks, is the answer to avoid.

Sources