Why it matters
Hiring managers cannot verify claims from a CV, so they look for artefacts that show how someone thinks. A demo shows that something ran once. A portfolio project shows the decisions behind it: why this retrieval unit, what the evaluation measured, what broke and what changed. The difference is not size, it is whether a reader can see judgement without interviewing you.
Mental model
A portfolio project is a lab notebook, not a trophy. The notebook shows the experiment, the failed attempt and the revised hypothesis. The metaphor breaks because a notebook is private, while a portfolio is read by someone deciding whether to spend an hour interviewing you.
How it works
- 1Pick a problem small enough to finish in a few weeks and specific enough that success can be measured.
- 2Write the evaluation before the feature, with a small labelled set and one metric that matters.
- 3Build the simplest version that can be measured, and record the baseline.
- 4Keep a decision log with the alternatives you rejected and why, including the ones you got wrong.
- 5Write a README that states the problem, the approach, the results and the known limitations in that order.
- 6Add one failure analysis section with a concrete case the system handles badly.
Step 1 of 6
Pick a problem small enough to finish in a few weeks and specific enough that success can be measured.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Pick a problem small enough to finish in a few weeks and specific enough that success can… 2. Write the evaluation before the feature, with a small labelled set and one metric that ma… 3. Build the simplest version that can be measured, and record the baseline. 4. Keep a decision log with the alternatives you rejected and why, including the ones you go… 5. Write a README that states the problem, the approach, the results and the known limitatio… 6. Add one failure analysis section with a concrete case the system handles badly.
- Pick a problem small enough to finish in a few weeks and specific enough that success can be measured.
- Write the evaluation before the feature, with a small labelled set and one metric that matters.
- Build the simplest version that can be measured, and record the baseline.
- Keep a decision log with the alternatives you rejected and why, including the ones you got wrong.
- Write a README that states the problem, the approach, the results and the known limitations in that order.
- Add one failure analysis section with a concrete case the system handles badly.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Narrow and deep | Shows judgement and rigour | Looks small next to a platform demo | The role values breadth of shipped features |
| Broad platform | Demonstrates many components | Shallow evaluation and unclear ownership | A reviewer cannot tell what you decided |
| Reproduce a known system | Comparable baseline | Little original judgement | The role needs novel design |
| Original problem | Distinctive and motivating | No baseline to compare against | You cannot evaluate it credibly |
Good for
Shows judgement and rigour
Cost
Looks small next to a platform demo
Fails when
The role values breadth of shipped features
- The role values breadth of shipped features
- A reviewer cannot tell what you decided
- The role needs novel design
- You cannot evaluate it credibly
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Narrow and deep is good for Shows judgement and rigour, costs Looks small next to a platform demo, and fails when The role values breadth of shipped features. Broad platform is good for Demonstrates many components, costs Shallow evaluation and unclear ownership, and fails when A reviewer cannot tell what you decided. Reproduce a known system is good for Comparable baseline, costs Little original judgement, and fails when The role needs novel design. Original problem is good for Distinctive and motivating, costs No baseline to compare against, and fails when You cannot evaluate it credibly
Failure modes
- Demo-only repository. The README shows a screenshot and no measurement. Debug: check whether a reader can find the metric and the test set in under a minute.
- Unfinished scope. The repository has scaffolding for five features and none complete. Debug: list what a user can do end to end today.
- No limitations. The write-up claims success everywhere, which reads as untested. Debug: search the README for a section describing a case that fails.
In production
Keep the project runnable with a seed script and a fixed evaluation set, so a reviewer can reproduce the number. Pin dependencies and record the model version used for the results.
Common misconceptions
People think: A portfolio needs to be a startup.
Actually: It needs to demonstrate judgement on a problem small enough to evaluate honestly.
People think: More projects signal more ability.
Actually: Several shallow projects dilute the evidence; one with a measured result and a limitation is easier to assess.
Interview angle
The strongest account of a project names the problem, the measurement, one decision and one failure, and is honest about limits. Weak answers demo features and cannot say what the evaluation showed.
Sources
- Will Larson, Staff Engineer guides — demonstrating judgement beyond task execution.
- Anthropic, Building effective agents — scoping a system small enough to evaluate.
- interviewing.io blog — what interviewers take from a candidate's artefacts.