For Developers/Glossary/Chain-of-Thought (CoT)
Prompting

Chain-of-Thought (CoT)

A prompting technique where a model is instructed to produce intermediate reasoning steps before generating a final answer, improving performance on complex tasks like math, logic, and code generation.

Chain-of-thought (CoT) prompting instructs a model to produce a sequence of reasoning steps before committing to a final answer. Instead of jumping directly to a conclusion, the model generates intermediate steps that build toward the answer. This approach consistently improves performance on tasks requiring multi-step reasoning: math problems, logical puzzles, code generation, and analysis.

How it works

The standard approach is adding "Let's think step by step" to the prompt, or demonstrating reasoning examples via few-shot examples. The model produces intermediate steps that build toward the answer, rather than jumping directly to a conclusion.

Why it helps

LLMs are next-token predictors. Without explicit reasoning tokens, the model must compress all its reasoning into the single probability distribution that predicts the answer token. With chain-of-thought, the model has working memory in the form of generated tokens to carry state across reasoning steps. This is especially valuable for arithmetic, multi-hop questions, and anything requiring tracking multiple constraints.

Zero-shot vs few-shot CoT

  • Zero-shot CoT: "Think step by step." No examples needed. Works well on modern frontier models.
  • Few-shot CoT: Provide 3-8 solved examples showing the reasoning process. Higher quality but requires more tokens and prompting effort.

Built-in reasoning in current models

Reasoning models, popularized by OpenAI's o1 in 2024, apply chain-of-thought internally at inference time. They generate reasoning tokens before the final answer, using additional compute at test time rather than only at training time. Those tokens are billed as output even when you cannot see them. Current flagship APIs handle this differently:

  • Claude: Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 use adaptive thinking. Thinking is on by default, the model decides how much to think on each request, and an effort setting controls depth. The older fixed budget_tokens mode is rejected on Claude 4.7 and later models. The API never returns the raw chain of thought: on current models the thinking text is omitted unless you request a summary with display: "summarized".
  • GPT-6: OpenAI's Responses API takes a reasoning.effort setting. GPT-6 Astra and GPT-6.1 Sol accept low, medium (the default), high, xhigh and max; GPT-6.1 Terra, GPT-6 Luna and the older GPT-6 Sol also accept none. Raw reasoning tokens are not visible, but you can request a reasoning summary.
  • Gemini 4 Argon: a thinking_level of low, medium (the default) or high sets how much the model reasons, and thought summaries can be returned alongside the answer.

Because these models already reason before answering, adding "think step by step" matters less than it did for earlier models. Explicit chain-of-thought prompting remains useful for models without built-in reasoning, or when you want the reasoning written out in the visible answer.

Related terms

Models relevant to Chain-of-Thought (CoT)