OpenAI has rolled out a feature named Structured Outputs in its API. This addition ensures that the model-generated outputs adhere to JSON Schemas provided by developers, improving the consistency of the results.
To use Structured Outputs, developers need to set `strict: true` within their function definitions. This feature works with models like GPT-4-0613 and GPT-3.5-turbo-0613 and later updates. Developers also have the option to supply a JSON Schema via the `json_schema` option in the `response_format` parameter. The latest GPT-4o models, including GPT-4o-2024-08-06 and gpt-4o-mini-2024-07-18, support this feature.
Improved Developer Tools
OpenAI has integrated Structured Outputs into its Python and Node SDKs. This streamlines the process for developers by managing JSON schema conversions and automatically parsing refusals. It facilitates easier creation of applications that can generate code or user interfaces based on inputs.
In line with OpenAI's safety policies, the Structured Outputs feature has mechanisms to refuse unsafe requests. The API responses include a new refusal string value, enabling developers to identify when a refusal occurs instead of producing outputs matching the schema.
Technical Approach
The implementation relies on a method known as constrained sampling or decoding, ensuring outputs align with supplied schemas. This involves converting JSON Schemas into context-free grammars (CFGs) and dynamically limiting the model's token options during sampling. Structured Outputs supports a subset of JSON Schema, and the first API response tied to a new schema may experience a latency due to preprocessing.
Structured Outputs can be used for extracting particular information, such as tasks, deadlines, and assignments from meeting notes. The model can also provide a separate field for a chain of thought, enhancing the clarity and quality of responses. However, this feature does not support parallel function calls and is excluded from Zero Data Retention policies.
Availability and Cost Benefits
The Structured Outputs feature is generally accessible via the API. It can be utilized with all models that support function calling in the API. The response formats with Structured Outputs are available on gpt-4o-mini and gpt-4o-2024-08-06 models. Notably, developers using gpt-4o-2024-08-06 can benefit from a 50% cost reduction on inputs and a 33% reduction on outputs compared to gpt-4o-2024-05-13.
