Skip to content
AI Engineer Guide.
juniorReportedcurated

What does temperature actually do, give the formula, and why is temperature 0 not a determinism guarantee?

30-second answer

Likely follow-ups

Open a question to rehearse the answer.

What temperature would you use for an LLM-as-judge and why?
  • Near zero, because the same input should get the same verdict and the judge is being used as a measurement rather than a generator.
  • A judge that varies between runs makes every eval comparison noisy, and the noise can hide a real regression or invent one.
  • Expect repetition and brittle phrasing as the cost, and pin the prompt version and schema so verdicts stay comparable across runs.
  • Verify it against human labels on a sample, since a consistent judge can be consistently wrong.
How does temperature interact with top-p ordering?
  • Temperature and truncation compose in the provider's documented order: if temperature runs first, flattening the distribution makes a fixed top-p keep more candidates; if truncation runs on raw logits, temperature changes the draw but not the candidate set.
  • At high temperature more mass sits in the tail, so a fixed top-p keeps a larger candidate set; at low temperature the same p keeps very few tokens.
  • Providers apply these in their own documented order and some reject the parameters entirely, so the same numeric pair is not portable across APIs.
  • Changing both at once makes a regression impossible to attribute, so move one knob per experiment.

Red flags

  • Says temperature changes which tokens are ranked highest
  • Asserts T=0 gives byte-identical outputs across calls

Concepts this tests

Tags

Source: github.com

More in LLM & transformer fundamentals