Introduction to Structured Outputs

• New API Feature: Structured Outputs ensure that model outputs adhere strictly to JSON Schemas provided by developers.
• Improved Reliability: The previous JSON mode did not guarantee schema conformity. The new Structured Outputs feature corrects this.

Use Cases

• Generating structured data from unstructured inputs.
• Building reliable applications like data extraction, user interface generation, and multi-step workflows.
• Examples: Extracting data for entries, separating final answers from reasoning, and dynamically generating UIs based on user input.

Function calling and response formats

• Function calling: Structured Outputs can be enabled by setting strict: true in function definitions. This is compatible with models gpt-4-0613 and later.
• Response Format: Developers can enforce a JSON Schema via the json_schema option in the response_format parameter. This applies to models like gpt-4o-2024-08-06 and gpt-4o-mini-2024-07-18.

Under the hood

• Training and Engineering: The GPT-4o-2024-08-06 model understands complex schemas due to training. Further, constrained decoding ensures strict schema adherence.
• Constrained Decoding: Restricting the model to valid tokens based on the JSON Schema ensures output correctness.
• Context-Free Grammar (CFG): Used for preprocessing JSON Schemas to facilitate efficient sampling, ensuring adherence to complex and nested schemas.

Limitations and restrictions

• Latency: The first response with a new schema incurs additional latency, but subsequent ones are quick.
• Errors: The accuracy of filled values within the JSON object isn't guaranteed.
• Parallel Calls: Structured Outputs are not compatible with parallel function calls.
• Safety Measures: Feature is aligned with OpenAI's safety standards to ensure no unsafe content is generated.