Chain-of-Thought (CoT)
A prompting technique where a model is instructed to produce intermediate reasoning steps before generating a final answer, improving performance on complex tasks like math, logic, and code generation.
Chain-of-thought (CoT) prompting instructs a model to produce a sequence of reasoning steps before committing to a final answer. Instead of jumping directly to a conclusion, the model generates intermediate steps that build toward the answer. This approach consistently improves performance on tasks requiring multi-step reasoning: math problems, logical puzzles, code generation, and analysis.
How it works
The standard approach is adding "Let's think step by step" to the prompt, or demonstrating reasoning examples via few-shot examples. The model produces intermediate steps that build toward the answer, rather than jumping directly to a conclusion.
Why it helps
LLMs are next-token predictors. Without explicit reasoning tokens, the model must compress all its reasoning into the single probability distribution that predicts the answer token. With chain-of-thought, the model has working memory in the form of generated tokens to carry state across reasoning steps. This is especially valuable for arithmetic, multi-hop questions, and anything requiring tracking multiple constraints.
Zero-shot vs few-shot CoT
- Zero-shot CoT: "Think step by step." No examples needed. Works well on modern frontier models.
- Few-shot CoT: Provide 3-8 solved examples showing the reasoning process. Higher quality but requires more tokens and prompting effort.
Built-in reasoning in current models
Reasoning models, popularized by OpenAI's o1 in 2024, apply chain-of-thought internally at inference time. They generate reasoning tokens before the final answer, using additional compute at test time rather than only at training time. Those tokens are billed as output even when you cannot see them. Current flagship APIs handle this differently:
- Claude: Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 use adaptive thinking. Thinking is on by default, the model decides how much to think on each request, and an
effortsetting controls depth. The older fixedbudget_tokensmode is rejected on Claude 4.7 and later models. The API never returns the raw chain of thought: on current models the thinking text is omitted unless you request a summary withdisplay: "summarized". - GPT-6: OpenAI's Responses API takes a
reasoning.effortsetting. GPT-6 Astra and GPT-6.1 Sol accept low, medium (the default), high, xhigh and max; GPT-6.1 Terra, GPT-6 Luna and the older GPT-6 Sol also accept none. Raw reasoning tokens are not visible, but you can request a reasoning summary. - Gemini 4 Argon: a
thinking_levelof low, medium (the default) or high sets how much the model reasons, and thought summaries can be returned alongside the answer.
Because these models already reason before answering, adding "think step by step" matters less than it did for earlier models. Explicit chain-of-thought prompting remains useful for models without built-in reasoning, or when you want the reasoning written out in the visible answer.
Related terms
Models relevant to Chain-of-Thought (CoT)
Claude Opus 5.5
Anthropic's current Opus and its recommended starting model, priced at $4/$20 per 1M tokens
View model →GPT-6 Astra
OpenAI's most capable model for the most demanding work
View model →Gemini 4 Argon
Google's frontier model for real-world coding, enterprise knowledge work, and cybersecurity defense
View model →