Chain-of-Thought (CoT)

Chain-of-thought (CoT) prompting instructs a model to produce a sequence of reasoning steps before committing to a final answer. Instead of jumping directly to a conclusion, the model generates intermediate steps that build toward the answer. This approach consistently improves performance on tasks requiring multi-step reasoning: math problems, logical puzzles, code generation, and analysis.

How it works

The standard approach is adding "Let's think step by step" to the prompt, or demonstrating reasoning examples via few-shot examples. The model produces intermediate steps that build toward the answer, rather than jumping directly to a conclusion.

Why it helps

LLMs are next-token predictors. Without explicit reasoning tokens, the model must compress all its reasoning into the single probability distribution that predicts the answer token. With chain-of-thought, the model has working memory in the form of generated tokens to carry state across reasoning steps. This is especially valuable for arithmetic, multi-hop questions, and anything requiring tracking multiple constraints.

Zero-shot vs few-shot CoT

• Zero-shot CoT: "Think step by step." No examples needed. Works well on modern frontier models.
• Few-shot CoT: Provide 3-8 solved examples showing the reasoning process. Higher quality but requires more tokens and prompting effort.

Internal reasoning in modern models

Newer models like OpenAI's o1, DeepSeek R1, and Claude's extended thinking mode apply chain-of-thought reasoning internally at inference time. These models generate reasoning tokens during inference to improve quality on difficult tasks, using additional compute at test time rather than only at training time. For users, this approach often produces higher quality answers on complex reasoning tasks compared to standard CoT prompting, though the internal reasoning steps may not always be visible.