Temperature
A sampling parameter that controls randomness in an LLM's output by scaling the probability distribution of tokens before sampling - lower values produce more deterministic results, higher values produce more varied results.
When an LLM generates the next token, it produces a probability distribution over its entire vocabulary. Temperature scales this distribution before sampling: low temperatures sharpen the distribution (the most likely token becomes overwhelmingly favored), while high temperatures flatten it (unlikely tokens get more chances to appear).
Temperature values in practice
- 0: Greedy decoding. Always picks the highest-probability token. Output is deterministic. Best for factual extraction, code generation, and structured data.
- 0.1 to 0.3: Near-deterministic with small variation between runs. Suitable for most production tasks where consistency matters.
- 0.7 to 1.0: Noticeable variation across outputs. Useful for creative writing, brainstorming, and cases where diversity is desired.
- Above 1.0: Very high randomness. Output quality degrades quickly. Rarely useful in practice.
Temperature and token sampling
Temperature works together with top-p (nucleus) sampling and top-k sampling. A common production configuration is temperature=0.2 with top-p=0.95 for general tasks, providing slight output variation while avoiding degenerate low-probability tokens. Some models also support min-p sampling as an alternative constraint.
Temperature does not determine quality
A common misconception is that higher temperature equals lower quality and lower temperature equals higher quality. The correct temperature depends on your task. Code output at temperature=1.0 is unreliable; creative writing at temperature=0 is robotic. Match temperature to your specific use case.