Why it matters
An LLM is the component that turns an application's intent into language. It is also the component that behaves differently on Tuesday. Understanding what the model actually is — a conditional distribution over tokens, not a database lookup and not a reasoning engine by construction — explains most of the engineering work around it: retrieval, evaluation, schema validation, guardrails, and cost control.
The practical consequence is that an LLM application is a probabilistic system. You cannot read the source of an answer. You can only constrain the inputs, sample deterministically where possible, validate the output, and measure the result. Teams that treat the model as an oracle ship brittle products; teams that treat it as a capable but unreliable function ship systems around it.
Mental model
Think of a very well-read autocomplete that has read a large fraction of the public internet. Given a prefix, it produces a probability for every token in its vocabulary, and generation walks that distribution one token at a time. The "knowledge" is compressed into weights during training; the "reasoning" is whatever computation the forward pass performs over the tokens you supplied.
The analogy breaks in two places. First, post-training changes the distribution substantially: an instruction-tuned model is not just a raw text continuer, it has been shaped to answer, refuse and format. Second, the model has no memory between calls. Any continuity you observe comes from an application re-sending history or storing state outside the model.
How it works
- 1Tokenise the prompt into vocabulary IDs, including special tokens that mark roles and boundaries.
- 2Embed the IDs, add positional information, and pass the sequence through transformer layers.
- 3At the final position, project the hidden state onto the vocabulary to produce logits.
- 4Turn logits into probabilities, apply a decoding rule, and select one token.
- 5Append the token and repeat until an end condition is met, then detokenise.
Pretraining optimises the same objective at every position: predict the next token. The model never sees a supervised label for "what a good answer looks like" during this stage. That is why pretraining produces a general text model and why it will happily continue a prompt with more prompt-like text rather than answer it.
Post-training supplies the assistant behaviour. Instruction tuning trains on demonstrations of instructions and responses. Preference optimisation — RLHF or a direct method such as DPO — shifts the distribution toward responses people prefer. The result is a model that follows requests far more reliably, at the cost of a narrower behavioural range and new failure modes such as over-refusal.
At inference time the model is a pure function of its input tokens and its weights. Temperature and truncation decide how the distribution is sampled. The context window bounds how many tokens can be considered at once. The KV cache makes generation affordable by avoiding recomputation, but it is an implementation detail of serving, not something the model "remembers" semantically.
Quick facts
- Input unit
- Tokens, not words
- State
- Stateless between calls
- Output
- A distribution over the vocabulary
- Training
- Pretraining, then post-training
- Main knobs
- Context, sampling, tools, schema
- Guarantee
- None on factuality
The objective, in one line
Training maximises the likelihood of the observed text under an autoregressive factorisation:
Every capability the product relies on — summarising, extracting, classifying, calling a tool — is expressed as a conditional continuation of this form. The vocabulary is fixed at training time, which is why tokenisation and the context window leak into every cost and latency estimate.
The generation loop
Every provider SDK wraps roughly this loop. It is shown as plain pseudocode because the important part is the shape, not a vendor signature.
# Illustrative: the loop that every hosted API hides behind a streaming response.
def generate(prompt_ids: list[int], model, max_new_tokens: int = 256) -> list[int]:
ids = list(prompt_ids)
for _ in range(max_new_tokens):
logits = model.forward(ids)[-1] # one score per vocabulary item
next_id = sample(softmax(logits / temperature))
if next_id == model.eos_id:
break
ids.append(next_id)
return ids[len(prompt_ids):]Two properties of this loop matter operationally. Cost and latency scale with the number of output tokens, because each one requires a full forward pass. And the model cannot revise what it has already emitted, so early mistakes propagate — which is why structured output, verification and retries exist as separate layers.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Frontier hosted model | Hard reasoning, broad coverage, fastest start | Highest per-token price, vendor dependency | The task is narrow, high-volume, or latency-critical |
| Smaller hosted model | Classification, extraction, routing | Lower price, weaker on multi-step tasks | The task needs judgement or long context |
| Open-weights model, self-hosted | Data control, fine-tuning, predictable volume | GPU capacity and operations | Traffic is spiky or the team has no serving experience |
| Fine-tuned small model | Stable, repetitive format-heavy tasks | Training data and maintenance | Requirements change faster than you can retrain |
Good for
Hard reasoning, broad coverage, fastest start
Cost
Highest per-token price, vendor dependency
Fails when
The task is narrow, high-volume, or latency-critical
- The task is narrow, high-volume, or latency-critical
- The task needs judgement or long context
- Traffic is spiky or the team has no serving experience
- Requirements change faster than you can retrain
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Frontier hosted model is good for Hard reasoning, broad coverage, fastest start, costs Highest per-token price, vendor dependency, and fails when The task is narrow, high-volume, or latency-critical. Smaller hosted model is good for Classification, extraction, routing, costs Lower price, weaker on multi-step tasks, and fails when The task needs judgement or long context. Open-weights model, self-hosted is good for Data control, fine-tuning, predictable volume, costs GPU capacity and operations, and fails when Traffic is spiky or the team has no serving experience. Fine-tuned small model is good for Stable, repetitive format-heavy tasks, costs Training data and maintenance, and fails when Requirements change faster than you can retrain
Failure modes
Confident fabrication. The model produces a fluent claim with no source. Debug by separating retrieval failure from generation failure: if the supporting passage is absent from the prompt, no prompt change will fix it. Add an abstain path and check it against questions whose answers are known to be missing.
Silent context loss. The prompt exceeds the window or the output hits the token cap, and the answer is truncated. Debug by logging the counted prompt tokens, the output token count and the finish reason for every call. A finish reason of "length" is a bug in the budget, not in the model.
Non-reproducible behaviour. The same input yields different output after a provider upgrade, or even across identical calls. Debug by pinning model versions where the provider allows it, fixing sampling parameters, and treating any prompt change as a release with an eval run.
Stale knowledge. The model answers from training data when the current answer differs. Debug by checking whether the question is time-sensitive; if it is, route it to retrieval or a tool and require the answer to cite the fresh source.
Symptom
The model produces a fluent claim with no source.
Debug step
Separating retrieval failure from generation failure: if the supporting passage is absent from the prompt, no prompt change will fix it. Add an abstain path and check it against questions whose answers are known to be missing.
Figure. Each failure with its symptom and the check that would confirm it.
What to notice. A failure without a debug step is a rumour. Pick the symptom you are seeing; the check is the next action.
Text version
Failure modes: Confident fabrication: The model produces a fluent claim with no source. Debug by separating retrieval failure from generation failure: if the supporting passage is absent from the prompt, no prompt change will fix it. Add an abstain path and check it against questions whose answers are known to be missing. | Silent context loss: The prompt exceeds the window or the output hits the token cap, and the answer is truncated. Debug by logging the counted prompt tokens, the output token count and the finish reason for every call. A finish reason of "length" is a bug in the budget, not in the model. | Non-reproducible behaviour: The same input yields different output after a provider upgrade, or even across identical calls. Debug by pinning model versions where the provider allows it, fixing sampling parameters, and treating any prompt change as a release with an eval run. | Stale knowledge: The model answers from training data when the current answer differs. Debug by checking whether the question is time-sensitive; if it is, route it to retrieval or a tool and require the answer to cite the fresh source.
In production
Treat the model call as an external dependency with a cost, a latency distribution and a failure rate. Log the model identifier and version, token counts, sampling parameters, finish reason, and a hash of the rendered prompt. Monitor tokens per request and per user, p95 latency, error and retry rates, and refusal rate; each moves before a user complaint does. Keep a small evaluation set that runs on every prompt or model change, because "it looks better" is not evidence.
Common misconceptions
People think: The model looks facts up in a database inside its weights.
Actually: Weights encode statistical patterns from training. There is no lookup table and no provenance, which is why citation and retrieval have to be added by the application.
People think: A larger context window means the model remembers more.
Actually: A larger window allows more input tokens. Whether the model uses them well depends on position, relevance and the task, and long inputs can degrade accuracy.
People think: Temperature zero makes output deterministic.
Actually: It makes sampling greedy, but batching, hardware nondeterminism and provider-side changes can still produce variation. Reproducibility is a systems property, not a parameter.
Interview angle
Interviewers use "what is an LLM" as a filter for whether you understand the system boundary. A strong answer names the training objective, distinguishes pretraining from post-training, states that inference is stateless, and connects those facts to the engineering consequences: token budgets, retrieval, evaluation and validation. A weak answer lists product features. Practise with the LLM fundamentals questions and the prompting bank.
Sources
- Vaswani et al., Attention Is All You Need — the transformer architecture underlying every model discussed here.
- Brown et al., Language Models are Few-Shot Learners — scale, in-context learning, and the limits of the pretraining-only view.
- Ouyang et al., Training language models to follow instructions with human feedback — instruction tuning and preference optimisation.
- Bommasani et al., On the Opportunities and Risks of Foundation Models — the foundation-model framing and its risks.
Related
- Inference — how the forward pass becomes a served product.
- Tokens — the unit of cost, context and truncation.
- Context budget — allocating the window.
- Decoding — turning logits into text.
- Transformer — the architecture in detail.