Skip to content
AI Engineer Guide.
seniorReportedcurated

How do you ingest a 200-page PDF of charts and tables where the answer exists only inside a line chart, with no matching text?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

How do you evaluate caption quality when a vision model invents a plausible number for an axis value?
  • Render the figure and check the caption's claims against it on a sample, since a wrong caption makes the figure retrievable for claims it does not support.
  • Test with figures whose values you can confirm from a source table, so an invented axis value is detectable rather than plausible.
  • Ask for captions that quote axis labels verbatim, and reject captions stating values the image does not print.
Would you index the page image, the caption, or both, and what does each cost at query time?
  • Index the caption for matching and keep the image reference for generation, which separates retrieval from what the model reads.
  • Page-image embedding matches layout-heavy questions, but its token cost is paid when the page image is created at ingest and again when it is attached to a query, so reserve it for pages where layout is the content.
  • Attaching images to the prompt spends prefill tokens and time to first token, so cap images per request and cache rendered pages.
How do you keep chart captions fresh when the source PDF is re-issued with revised figures?
  • Key the caption on document version plus a figure hash, and regenerate only the figures whose bytes changed.
  • Store the caption with its source span and bounding box, so a revised figure can be traced and its old caption retired.
  • Propagate revisions through the same change feed as text, or the old caption keeps answering after the PDF is re-issued.

Red flags

  • assumes OCR recovers chart data
  • misses unindexed content

Concepts this tests

Tags

Source: www.reddit.com

More in Data & storage