Skip to content
AI Engineer Guide.
midReportedcurated

Take-home: build a RAG assistant over a document set with citations. What do you build first, and how do you scope it?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

What did you deliberately leave out, and what would break first at 100x documents?
  • I left out authentication, per-tenant filtering and UI polish, because none of them change whether the answers are grounded.
  • At 100x documents, ingestion idempotency and deletion propagation break first, since a re-sync that only adds leaves stale versions retrievable.
  • Retrieval latency and re-embedding cost come next, which is why the scope note names them as the next engineering steps rather than hiding them.
How did you use AI tools, and where did you reject their output?
  • I used them for scaffolding, boilerplate and a first draft of the README, which is work that does not carry a design decision.
  • I rejected generated eval questions that the retriever found easy, because a set written by the same system that answers it measures the wrong thing.
  • I also rejected any citation the code could not resolve to retrieved text, since an unverifiable citation is worse than no citation.

Red flags

  • Builds a polished UI with no evaluation harness
  • Cannot explain why the chunk size or top-k was chosen

Concepts this tests

Tags

Source: github.com

More in Take-homes & career