Temperature (LLM)

What does turning temperature down actually buy?

Consistency of style and structure, mostly, and less than people expect of substance. Anthropic’s guidance is to use temperature “closer to 0.0 for analytical / multiple choice, and closer to 1.0 for creative and generative tasks”, which is a statement about the shape of the output rather than about its accuracy. A low temperature does not make a model more correct; it makes it less varied. That distinction matters when somebody proposes turning temperature down to make an assistant stop saying something wrong about a brand.

It will make the wrong thing more consistent, which is the opposite of the intended effect, and the actual fix is on the retrieval side. It matters in the other direction too, for anyone building on an API: a low temperature does not make a summarisation task safe to leave unchecked, it makes its errors repeatable. Repeatable errors are easier to find, which is a genuine argument for low temperature in a pipeline and not an argument for trusting the output.

Why does zero temperature still not repeat itself?

Anthropic states it directly: “note that even with temperature of 0.0, the results will not be fully deterministic”. The reasons sit below the parameter, in floating-point arithmetic that does not associate the same way across different hardware and batch sizes, and in serving infrastructure that changes underneath a stable API.

This matters far outside the API, because it is the floor under every AI visibility measurement. If the model itself cannot be made to repeat exactly, then a single answer is a sample rather than a reading, and any tool reporting a brand’s position in one AI response is reporting one draw from a distribution as though it were a rank.

Does temperature explain why AI answers about your brand vary?

Partly, and it is the smaller half. Temperature varies the wording; retrieval varies the substance. An assistant answering the same question twice may search differently, get different pages back and cite different sources, and that churn is larger than anything the sampling parameter contributes. Ahrefs measured AI Overviews changing every 2.15 days on average across 43,000 keywords, with 45.5% of citations turning over when they do.

So the recommendation, which could be wrong for a research use case: do not spend time trying to pin an assistant down to a repeatable answer. Measure the distribution instead, across a fixed panel run repeatedly, and read the direction.

Related terms

• Token (LLM)A token is the unit a language model processes text in: a fragment that may be a whole word, part of one, a space or a punctuation mark.
• Context WindowA context window is all the text a language model can reference when generating a response, including the response itself.
• AI HallucinationAn AI hallucination is a statement a language model produces confidently and fluently that is not true.
• Citation RateCitation rate is the share of tracked AI answers that link your site as a source, counted over the answers where a source list was actually returned.