Skip to content
AI Engineer Guide.

pattern · architecture · intermediate · concept 7 of 7

Offline batch enrichment

0 of 7 complete

Why it matters

Enriching every record at write time is expensive and slow, and many enrichments are not needed immediately. A batch job processes records in bulk, uses larger requests and cheaper scheduling, and writes results back. The problem it creates is two speeds of data: freshly written records lack enrichment while the batch catches up, and a partial run leaves the corpus in mixed states.

Mental model

A laundry service that picks up and returns: cheap per item, but you wear something else while your clothes are in the van. The metaphor breaks because a mixed corpus does not look dirty; it looks normal until a query returns an unenriched record.

How it works

The batch job sits beside the request path, not in itInteractive
signalbypassloop

Step 1 of 5

Write first

The write path never waits for enrichment. From this moment the record is readable and unenriched, and the serving path has to handle that state.

Active layers

  • Record written
  • Record store

Hover a layer to read what it holds.

Next: Select a window

Freshness
lag behind writes
Key
record id + version
Resume
checkpoint per batch
Step 1/5 · Write first

Figure. Write and read paths meet at the record store, and the job hangs below the request path rather than sitting on it. A write lands unenriched; the job reads a window by watermark or hash and writes back a versioned row; a read can therefore see an old version, a new one, or none. Queueing, retries and the job's per-record prompts are omitted.

What to notice. Nothing on the request side waits for the job: the two lanes that leave it are the write-back into the store and the versioned row the read consults. The solid lane into the serving read is that versioned row; the dashed lane is a record the job has not reached, which is a normal state during every run rather than an error. Without the version in the key, a rerun cannot tell its own output from the previous run's.

Text version

A vertical stack. A written record, with enrichment absent, lands in the record store. The store feeds a serving read, which produces the answer, and the offline batch job group sits below the answer, outside the request path. A dashed lane runs from the store straight to the serving read, labelled not yet enriched. The store also feeds the job group: select a window by watermark or hash, enrich in bulk, and a versioned upsert keyed by record id plus version. The upsert writes back into the store along a dash-dot lane labelled write back, and commits to a versioned-rows band whose lane, labelled row + version, reaches the serving read. Five steps cover the write, window selection, bulk enrichment, the versioned write, and the two speeds of read.

  1. Define the enrichment contract: input fields, output fields, model or prompt version, and the schema of the stored result.
  2. Select work by a watermark or content hash, so reruns process only new or changed records.
  3. Write results with the enrichment version, and make the write an upsert keyed by record id and version.
  4. Checkpoint progress per batch, so a failure resumes instead of restarting the corpus.
  5. Keep a serving path for unenriched records, such as a fallback or an exclusion filter, instead of assuming the batch finished.
  6. Estimate backfill cost from record count and per-record tokens before changing the enrichment schema.

Trade-offs

ChoiceGood forCostFails when
Nightly full batchSimple and predictableReprocesses unchanged recordsThe corpus is large
Incremental by watermarkEfficientWatermark bugs skip recordsUpdates arrive out of order
Large batches per requestCheaper per recordOne failure loses more workRecords are heterogeneous
Enrich at write timeAlways freshCost and latency per writeTraffic is bursty
Trade-off mapInteractive

Good for

Simple and predictable

Cost

Reprocesses unchanged records

Fails when

The corpus is large

  1. The corpus is large
  2. Updates arrive out of order
  3. Records are heterogeneous
  4. Traffic is bursty

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Nightly full batch is good for Simple and predictable, costs Reprocesses unchanged records, and fails when The corpus is large. Incremental by watermark is good for Efficient, costs Watermark bugs skip records, and fails when Updates arrive out of order. Large batches per request is good for Cheaper per record, costs One failure loses more work, and fails when Records are heterogeneous. Enrich at write time is good for Always fresh, costs Cost and latency per write, and fails when Traffic is bursty

Failure modes

  • Mixed-version corpus. Half the records have old enrichment and queries cannot tell. Debug: count records per enrichment version and compare with the expected rollout.
  • Watermark gap. Records written during the job are skipped because the watermark advanced past them. Debug: compare record counts by creation time with processed counts for the same window.
  • Partial batch loss. The job fails after writing some results and the retry starts over or skips. Debug: count outputs per batch id and verify the checkpoint matches the last committed record.

In production

Monitor enrichment coverage per version, job duration and lag behind the newest record. Alert when coverage drops or when the watermark falls behind by more than a defined window.

Common misconceptions

People think: Batch enrichment is just a cron job.

Actually: It is a resumable, versioned data pipeline; the cron is the least important part.

People think: Staleness is acceptable because the batch runs nightly.

Actually: It is acceptable only if the serving path handles unenriched records explicitly.

Interview angle

Name the freshness requirement first, then the job's idempotency, watermark and versioning. Strong answers discuss the mixed-corpus problem and backfill cost. Weak ones describe a script that loops over rows.

Sources