Why it matters
Every AI product is assembled from models, infrastructure and orchestration. Building all of it wastes effort on solved problems; buying all of it leaves nothing distinctive and couples the product to a supplier's roadmap. The decision is per component, and the deciding question is whether this component is where your advantage lives or merely where it runs.
Mental model
Build versus buy is deciding which rooms to renovate and which to rent. The metaphor breaks because renting does not let a landlord change the walls while you sleep, while a hosted component can change behaviour under a stable name, so the exit plan matters more than the metaphor suggests.
How it works
Step 1 of 4
One component at a time
Decompose into model access, orchestration, retrieval, evaluation, observability and interface. A component that is not where the advantage lives starts in the buy corner.
Differentiation: low · Operating cost: low
Choose · Buy hosted
The supplier already amortises the work; keep one thin interface at the boundary.
Serving, tracing, storage and generic model access usually sit here. Prototype the buy option first: it is faster and it reveals the requirements.
Axes
- Differentiation
- commodity → where the advantage lives
- Operating cost
- cheap to run → staffing and exit
Figure. A component's place is set by how much differentiation it provides and how much operating cost it commits you to. The same hosted supplier is right in the commodity corner and wrong where it is the advantage.
What to notice. The axes are ordered, not scored: the figure does not price a component or weigh its advantage. Data terms, retention and exit cost are conditions on every cell, not a tie-breaker after the decision. “Operating cost” means total cost of ownership — licence, integration, on-call, upgrades and migration — not the monthly bill.
Text version
A two-by-two decision map. The horizontal axis is differentiation, from commodity on the left to where the advantage lives on the right. The vertical axis is operating cost, from cheap to run at the bottom to staffing and exit at the top. Bottom left, commodity and cheap to run: buy hosted, because a mature supplier amortises the work and one thin interface keeps the exit cheap. Bottom right, the advantage at low cost: build thin and keep it swappable. Top left, no differentiation but a large recurring bill: read the exit cost, then renegotiate or replace, because the terms decide whether to stay or rebuild. Top right, the thing customers notice at real operating cost: build and own it, and staff the on-call. Four steps walk the cells in that order.
- Decompose the system into components: model access, orchestration, retrieval, evaluation, observability, interface.
- For each, ask whether it is a source of advantage or a commodity with a mature supplier.
- Estimate total cost of ownership: licence, integration, on-call, upgrades and the migration if you leave.
- Read the data terms: what is stored, for how long, and whether your inputs can train another party's model.
- Prototype the buy option first, since it is usually faster and reveals the requirements.
- Write the exit condition for each bought component before signing, including what would trigger a rebuild.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Buy hosted | Fast start, low operations | Recurring cost, less control, exit work | The component is core to differentiation |
| Build in-house | Control and fit to the workflow | Staffing, maintenance and opportunity cost | The team is small or the problem is generic |
| Open model, self-hosted | Portability and predictable cost | Inference operations and quality ownership | Traffic is spiky or volumes are low |
| Framework adoption | Conventions and speed | Abstraction leakage and upgrade churn | You need behaviour the framework does not model |
Good for
Fast start, low operations
Cost
Recurring cost, less control, exit work
Fails when
The component is core to differentiation
- The component is core to differentiation
- The team is small or the problem is generic
- Traffic is spiky or volumes are low
- You need behaviour the framework does not model
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Buy hosted is good for Fast start, low operations, costs Recurring cost, less control, exit work, and fails when The component is core to differentiation. Build in-house is good for Control and fit to the workflow, costs Staffing, maintenance and opportunity cost, and fails when The team is small or the problem is generic. Open model, self-hosted is good for Portability and predictable cost, costs Inference operations and quality ownership, and fails when Traffic is spiky or volumes are low. Framework adoption is good for Conventions and speed, costs Abstraction leakage and upgrade churn, and fails when You need behaviour the framework does not model
Failure modes
- Hidden exit cost. A supplier change forces a rewrite because the integration leaked everywhere. Debug: count the files that import the vendor SDK and check they sit behind one interface.
- Commodity build. Months spent rebuilding tracing or serving with no product gain. Debug: compare the component's roadmap with the supplier's release notes.
- Terms discovered late. Data retention terms block a customer deal after launch. Debug: add data terms to the architecture review checklist and verify them before the integration lands.
In production
Track spend and dependency per bought component, and review contracts before renewal. Keep one thin abstraction at each boundary so a replacement is a project rather than a rewrite.
Common misconceptions
People think: Building is always more control.
Actually: Control arrives with maintenance obligations; a small team often controls less because it cannot keep up.
People think: Buying is always faster.
Actually: Integration, data terms and review can take longer than a focused build of a well-understood component.
Interview angle
Strong answers decide per component, name where the differentiation sits, and include exit cost and data terms in the comparison. Weak answers frame it as a preference for owning code or for moving fast.
Sources
- The Twelve-Factor App, Backing services — treating external services as swappable attached resources.
- Sculley et al., Hidden Technical Debt in Machine Learning Systems — the maintenance cost that builds accumulate.
- Anthropic, Building effective agents — start with direct primitives before adopting a framework.