Structured Output
Constraining an LLM to return output that matches a defined format, usually a JSON Schema, so code can parse it directly instead of extracting values from free text.
LLMs generate free-form text by default. Structured output features make the model return output that conforms to a schema, so downstream code can parse it directly without regex extraction or cleanup.
Approaches
- JSON mode: The API guarantees syntactically valid JSON but does not enforce any particular structure.
- Schema-enforced output: You pass a JSON Schema and the API constrains the response to match it. In OpenAI's Responses API this is
text: { format: { type: "json_schema", strict: true, schema: {...} } }; Chat Completions usesresponse_formatwithtype: "json_schema". Anthropic's Claude API usesoutput_config.formatwithtype: "json_schema", Mistral accepts a schema throughresponse_format, and the Gemini API takes a schema alongside anapplication/jsonoutput type. - Strict tool use: Tool or function arguments are validated against the tool's input schema. OpenAI and Anthropic both accept
strict: trueon a function or tool definition. OpenAI requiresadditionalProperties: falseand every property listed as required, and its Responses API applies strict mode automatically when the schema allows it. - Constrained decoding: At each step, only tokens that keep the output valid are allowed. Anthropic says its structured outputs are built on constrained decoding, and libraries such as Outlines and Guidance, plus serving engines such as vLLM, provide it for self-hosted models.
Why schema-constrained output matters
Without it, you parse free text in production, and models sometimes change their output format between runs, especially at higher temperatures. Schema constraints remove that class of bug. The model still chooses the values; the schema only fixes the structure.
Limits of the guarantee
- Refusals and truncation: If the model refuses, or the response hits the output token limit, the result may be incomplete or not match the schema. OpenAI and Anthropic both document this, so check the stop reason or response status before parsing.
- Partial JSON Schema support: Providers support a subset of JSON Schema. Anthropic returns a 400 error for unsupported features such as numeric
minimumconstraints, while Gemini silently drops some constraints such asminLengthandmaxLength. - First-request latency: OpenAI and Anthropic process each new schema before first use, so the first request with a schema is slower; later requests with the same schema are not.
Practical usage
The official SDKs include helpers that turn a Pydantic or Zod model into a schema and parse the response back into a typed object, such as responses.parse in OpenAI's SDKs and messages.parse in Anthropic's. For Python code that has to work across providers, the Instructor library offers one Pydantic-based interface:
import instructor
from pydantic import BaseModel
class User(BaseModel):
name: str
email: str
client = instructor.from_provider("openai/gpt-6-luna")
user = client.create(
response_model=User,
messages=[{"role": "user", "content": "Extract: Ana Ruiz, ana@example.com"}],
)Even with strict schema enforcement, validate the values your code depends on: the schema guarantees the shape of the output, not that the extracted data is correct.
Related terms
Models relevant to Structured Output
GPT-6 Luna
OpenAI's cheapest GPT-6 model, and since October 2026 the default for free ChatGPT users
View model →Claude Haiku 5.5
Anthropic's fastest, lowest-cost current model at $0.10 per million input tokens with a 1M context window
View model →Gemini 3.8 Flash
Google's previous-generation Flash model for coding and agents, at half Gemini 4 Hydrogen's price until December 31, 2026
View model →