Why it matters
Enriching every record at write time is expensive and slow, and many enrichments are not needed immediately. A batch job processes records in bulk, uses larger requests and cheaper scheduling, and writes results back. The problem it creates is two speeds of data: freshly written records lack enrichment while the batch catches up, and a partial run leaves the corpus in mixed states.
Mental model
A laundry service that picks up and returns: cheap per item, but you wear something else while your clothes are in the van. The metaphor breaks because a mixed corpus does not look dirty; it looks normal until a query returns an unenriched record.
How it works
Step 1 of 5
Write first
The write path never waits for enrichment. From this moment the record is readable and unenriched, and the serving path has to handle that state.
Active layers
- Record written
- Record store
Hover a layer to read what it holds.
Next: Select a window
- Freshness
- lag behind writes
- Key
- record id + version
- Resume
- checkpoint per batch
Figure. Write and read paths meet at the record store, and the job hangs below the request path rather than sitting on it. A write lands unenriched; the job reads a window by watermark or hash and writes back a versioned row; a read can therefore see an old version, a new one, or none. Queueing, retries and the job's per-record prompts are omitted.
What to notice. Nothing on the request side waits for the job: the two lanes that leave it are the write-back into the store and the versioned row the read consults. The solid lane into the serving read is that versioned row; the dashed lane is a record the job has not reached, which is a normal state during every run rather than an error. Without the version in the key, a rerun cannot tell its own output from the previous run's.
Text version
A vertical stack. A written record, with enrichment absent, lands in the record store. The store feeds a serving read, which produces the answer, and the offline batch job group sits below the answer, outside the request path. A dashed lane runs from the store straight to the serving read, labelled not yet enriched. The store also feeds the job group: select a window by watermark or hash, enrich in bulk, and a versioned upsert keyed by record id plus version. The upsert writes back into the store along a dash-dot lane labelled write back, and commits to a versioned-rows band whose lane, labelled row + version, reaches the serving read. Five steps cover the write, window selection, bulk enrichment, the versioned write, and the two speeds of read.
- Define the enrichment contract: input fields, output fields, model or prompt version, and the schema of the stored result.
- Select work by a watermark or content hash, so reruns process only new or changed records.
- Write results with the enrichment version, and make the write an upsert keyed by record id and version.
- Checkpoint progress per batch, so a failure resumes instead of restarting the corpus.
- Keep a serving path for unenriched records, such as a fallback or an exclusion filter, instead of assuming the batch finished.
- Estimate backfill cost from record count and per-record tokens before changing the enrichment schema.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Nightly full batch | Simple and predictable | Reprocesses unchanged records | The corpus is large |
| Incremental by watermark | Efficient | Watermark bugs skip records | Updates arrive out of order |
| Large batches per request | Cheaper per record | One failure loses more work | Records are heterogeneous |
| Enrich at write time | Always fresh | Cost and latency per write | Traffic is bursty |
Good for
Simple and predictable
Cost
Reprocesses unchanged records
Fails when
The corpus is large
- The corpus is large
- Updates arrive out of order
- Records are heterogeneous
- Traffic is bursty
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Nightly full batch is good for Simple and predictable, costs Reprocesses unchanged records, and fails when The corpus is large. Incremental by watermark is good for Efficient, costs Watermark bugs skip records, and fails when Updates arrive out of order. Large batches per request is good for Cheaper per record, costs One failure loses more work, and fails when Records are heterogeneous. Enrich at write time is good for Always fresh, costs Cost and latency per write, and fails when Traffic is bursty
Failure modes
- Mixed-version corpus. Half the records have old enrichment and queries cannot tell. Debug: count records per enrichment version and compare with the expected rollout.
- Watermark gap. Records written during the job are skipped because the watermark advanced past them. Debug: compare record counts by creation time with processed counts for the same window.
- Partial batch loss. The job fails after writing some results and the retry starts over or skips. Debug: count outputs per batch id and verify the checkpoint matches the last committed record.
In production
Monitor enrichment coverage per version, job duration and lag behind the newest record. Alert when coverage drops or when the watermark falls behind by more than a defined window.
Common misconceptions
People think: Batch enrichment is just a cron job.
Actually: It is a resumable, versioned data pipeline; the cron is the least important part.
People think: Staleness is acceptable because the batch runs nightly.
Actually: It is acceptable only if the serving path handles unenriched records explicitly.
Interview angle
Name the freshness requirement first, then the job's idempotency, watermark and versioning. Strong answers discuss the mixed-corpus problem and backfill cost. Weak ones describe a script that loops over rows.
Sources
- Azure Architecture Center, Batch Processing — scheduling, checkpointing and reprocessing.
- Dean & Ghemawat, MapReduce: Simplified Data Processing on Large Clusters — bulk processing with restartable units of work.
- Kleppmann, Designing Data-Intensive Applications — batch versus stream processing and derived data.