For Developers/Glossary/Top-p (Nucleus Sampling)
Inference

Top-p (Nucleus Sampling)

A sampling strategy that selects tokens by including the most probable candidates until their cumulative probability reaches a threshold p, allowing the number of candidates to vary based on the model's confidence.

Top-p sampling, also called nucleus sampling, is an alternative to top-k sampling. Instead of keeping a fixed number of candidates (top-k), top-p includes only the most probable tokens whose combined probability reaches at least p. This adapts dynamically to the model's confidence: when the model is confident, only 2-3 tokens might qualify; when uncertain, dozens might.

Top-p vs top-k vs temperature

These three parameters influence token selection differently:

  • Temperature reshapes the probability distribution before sampling
  • Top-k restricts selection to the k most likely tokens
  • Top-p restricts selection to tokens whose cumulative probability reaches p

Where both are available, providers generally recommend adjusting temperature or top-p, not both. Top-k is less common in modern APIs; OpenAI's API does not offer it.

Common top-p values

  • 0.1: Very conservative; heavily favors the most likely completions
  • 0.9-0.95: A common setting that allows diversity without erratic outputs; Gemini models list 0.95 as their default
  • 1.0: No truncation; all tokens remain candidates. This is the default in OpenAI's API

Practical considerations

For most API use cases, the default top-p value requires no adjustment, and on many current flagship models it cannot be changed at all: Claude Fable 5.2, Claude Opus 5.5, Claude Sonnet 5.5 and Claude Haiku 5.5 return a 400 error for any non-default top_p, Gemini 4 Argon ignores it, and OpenAI's GPT-6 models accept it only when reasoning effort is set to none. Changing top-p alone rarely improves output quality significantly. When output quality is poor, examine the prompt and temperature settings first before modifying top-p. Different API providers may implement top-p sampling slightly differently, and some reject requests that set top-p and temperature together, so check the provider's documentation before setting both.

Related terms