For Developers/Glossary/Structured Output
Inference

Structured Output

Constraining an LLM to return output that matches a defined format, usually a JSON Schema, so code can parse it directly instead of extracting values from free text.

LLMs generate free-form text by default. Structured output features make the model return output that conforms to a schema, so downstream code can parse it directly without regex extraction or cleanup.

Approaches

  • JSON mode: The API guarantees syntactically valid JSON but does not enforce any particular structure.
  • Schema-enforced output: You pass a JSON Schema and the API constrains the response to match it. In OpenAI's Responses API this is text: { format: { type: "json_schema", strict: true, schema: {...} } }; Chat Completions uses response_format with type: "json_schema". Anthropic's Claude API uses output_config.format with type: "json_schema", Mistral accepts a schema through response_format, and the Gemini API takes a schema alongside an application/json output type.
  • Strict tool use: Tool or function arguments are validated against the tool's input schema. OpenAI and Anthropic both accept strict: true on a function or tool definition. OpenAI requires additionalProperties: false and every property listed as required, and its Responses API applies strict mode automatically when the schema allows it.
  • Constrained decoding: At each step, only tokens that keep the output valid are allowed. Anthropic says its structured outputs are built on constrained decoding, and libraries such as Outlines and Guidance, plus serving engines such as vLLM, provide it for self-hosted models.

Why schema-constrained output matters

Without it, you parse free text in production, and models sometimes change their output format between runs, especially at higher temperatures. Schema constraints remove that class of bug. The model still chooses the values; the schema only fixes the structure.

Limits of the guarantee

  • Refusals and truncation: If the model refuses, or the response hits the output token limit, the result may be incomplete or not match the schema. OpenAI and Anthropic both document this, so check the stop reason or response status before parsing.
  • Partial JSON Schema support: Providers support a subset of JSON Schema. Anthropic returns a 400 error for unsupported features such as numeric minimum constraints, while Gemini silently drops some constraints such as minLength and maxLength.
  • First-request latency: OpenAI and Anthropic process each new schema before first use, so the first request with a schema is slower; later requests with the same schema are not.

Practical usage

The official SDKs include helpers that turn a Pydantic or Zod model into a schema and parse the response back into a typed object, such as responses.parse in OpenAI's SDKs and messages.parse in Anthropic's. For Python code that has to work across providers, the Instructor library offers one Pydantic-based interface:

import instructor
from pydantic import BaseModel

class User(BaseModel):
    name: str
    email: str

client = instructor.from_provider("openai/gpt-6-luna")
user = client.create(
    response_model=User,
    messages=[{"role": "user", "content": "Extract: Ana Ruiz, ana@example.com"}],
)

Even with strict schema enforcement, validate the values your code depends on: the schema guarantees the shape of the output, not that the extracted data is correct.

Related terms

Models relevant to Structured Output