Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 51 of 57

Temporal retrieval

0 of 57 complete

Why it matters

Most corpora contain several versions of the same policy, price list or contract, and similarity search has no notion of which one applies. A question about last quarter's terms retrieves the current page because it is nearly identical and ranked slightly higher. Temporal retrieval makes time an explicit filter and an explicit attribute, so the answer can be tied to the period the question is about.

Mental model

Temporal retrieval is asking a library for the edition that was in force on a given date, not the newest edition on the shelf. The metaphor breaks because the library labels its editions clearly, while a re-uploaded document often looks like a new document with no relation to the old one.

How it works

Mechanism, step by stepInteractive
  1. 1Record two timestamps per document: when the content was true and when it entered the index.
  2. 2Model validity as an interval with an open end, and close it when a successor arrives rather than deleting the old row.
  3. 3Detect the temporal intent in the query: explicit date, relative period, or no temporal constraint at all.
  4. 4Apply time as a filter on the validity interval before similarity ranking, not as a post-hoc boost.
  5. 5When no date is given, prefer currently valid documents but keep superseded ones reachable with a flag.
  6. 6Show the validity period in the citation so the reader can see which version answered.

Step 1 of 6

Record two timestamps per document: when the content was true and when it entered the index.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Record two timestamps per document: when the content was true and when it entered the ind… 2. Model validity as an interval with an open end, and close it when a successor arrives rat… 3. Detect the temporal intent in the query: explicit date, relative period, or no temporal c… 4. Apply time as a filter on the validity interval before similarity ranking, not as a post-… 5. When no date is given, prefer currently valid documents but keep superseded ones reachabl… 6. Show the validity period in the citation so the reader can see which version answered.

  1. Record two timestamps per document: when the content was true and when it entered the index.
  2. Model validity as an interval with an open end, and close it when a successor arrives rather than deleting the old row.
  3. Detect the temporal intent in the query: explicit date, relative period, or no temporal constraint at all.
  4. Apply time as a filter on the validity interval before similarity ranking, not as a post-hoc boost.
  5. When no date is given, prefer currently valid documents but keep superseded ones reachable with a flag.
  6. Show the validity period in the citation so the reader can see which version answered.

Trade-offs

ChoiceGood forCostFails when
Validity intervalsCorrect as-of answersIngest must close intervals on updateUpdates arrive without an effective date
Recency boostSimple and no schema changeRewrites the question silentlyUsers ask about a past period
Current-only indexSmall and simpleHistorical questions are unanswerableThe domain is regulatory or contractual
Keep every versionFull history and auditNear-duplicate results and ranking ambiguityNo validity metadata distinguishes versions
Trade-off mapInteractive

Good for

Correct as-of answers

Cost

Ingest must close intervals on update

Fails when

Updates arrive without an effective date

  1. Updates arrive without an effective date
  2. Users ask about a past period
  3. The domain is regulatory or contractual
  4. No validity metadata distinguishes versions

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Validity intervals is good for Correct as-of answers, costs Ingest must close intervals on update, and fails when Updates arrive without an effective date. Recency boost is good for Simple and no schema change, costs Rewrites the question silently, and fails when Users ask about a past period. Current-only index is good for Small and simple, costs Historical questions are unanswerable, and fails when The domain is regulatory or contractual. Keep every version is good for Full history and audit, costs Near-duplicate results and ranking ambiguity, and fails when No validity metadata distinguishes versions

Failure modes

  • Two versions both marked current. The index contains an open interval for an old and a new document, and ranking picks arbitrarily. Debug: group by document family and assert exactly one open interval per family.
  • Ingestion time used as event time. A backfilled document appears to be from today and wins every recent query. Debug: compare the two timestamps for backfilled batches and look for clusters at the load date.
  • Temporal intent ignored. A question about a past quarter returns current figures with no warning. Debug: label queries with temporal intent and check whether a date filter was applied.

In production

Monitor the number of open intervals per document family and the share of queries with a temporal filter. Alert on documents whose validity interval closes without a successor.

Common misconceptions

People think: Sorting by date solves temporal questions.

Actually: Sorting finds the newest document, not the one that was valid at the asked-for date.

People think: Deleting old versions keeps the index clean.

Actually: It destroys the ability to answer historical questions and to audit what was true at the time.

Interview angle

Separate event time from ingestion time and model validity as intervals that are closed, not deleted. Filter before ranking and report the period in the citation. Boosting recent documents is the answer that silently rewrites the user's question.

Sources