Structured Output

LLMs generate free-form text by default. Structured output techniques force the model to produce output that conforms to a schema, making it directly parseable by downstream code without regex extraction or post-processing.

Approaches

• JSON mode: Tell the API to guarantee valid JSON output. No schema enforcement - the structure varies by prompt.
• JSON Schema: Provide a strict schema; the API guarantees the output matches the schema field types and required fields. OpenAI's response_format: { type: "json_schema", json_schema: {...} } is a standard implementation.
• Tool use: Some providers (Anthropic, OpenAI) support structured output through tool or function calling, where you define a schema as a tool parameter and the model selects tools with structured arguments.
• Constrained decoding: At the token level, only allow tokens that keep the output valid at each step. Libraries like Outlines and Guidance implement this client-side.

Why schema-constrained output matters

Without structured output, you parse free text in production. Models sometimes vary their output format between runs, especially at higher temperatures. Schema constraints eliminate this class of bug. The model still decides the values; the schema only constrains the structure.

Practical usage

Most major LLM providers now support structured output natively. For Python development, the Instructor library provides a Pydantic-based interface that works across providers (OpenAI, Anthropic, others) and simplifies schema definition. Example:

from instructor import Instructor
from pydantic import BaseModel

class User(BaseModel):
name: str
email: str

user = client.chat.completions.create(
model="gpt-4",
response_model=User,
messages=[...]
)

For complex nested schemas or when integrating with existing APIs, define JSON Schema directly in your request. This guarantees the model output will deserialize correctly without validation errors.