459 words
2 minutes
LLM Structured Outputs
What Is Structured Output?
Normally an LLM answers in natural language. But sometimes what you actually need is a specific structured payload — JSON, XML, and so on.
Structured output means the model returns data with a clean, well-defined shape instead of a wall of prose.
How to Use Structured Output
1. Specify the output format in the prompt
import OpenAI from 'openai';
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY,});
const completion = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'You are a professional historian, capable of summarizing a person\'s life concisely and accurately.', }, { role: 'user', content: `Summarize the 10 most important events in ${name}'s life. Output as JSON with these fields: { "name": "string", // name "gender": "string", // gender "type": "string", // type "profile": "string", // profile 100 words "timeline": [ { "year": "string", // year "title": "string", // title "description": "string", // description 50 words } ] } `, }, ],});
// Expected output```json{ "name": "string", // name "gender": "string", // gender "type": "string", // type "profile": "string", // profile 100 words "timeline": [ { "year": "string", // year "title": "string", // title "description": "string", // description 50 words } ]}2. Define the structure with zod, paired with OpenAI’s zodResponseFormat
import OpenAI from 'openai';import { zodResponseFormat } from 'openai/helpers/zod';import { z } from 'zod';
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY,});
const BiographySchema = z.object({ name: z.string(), gender: z.string(), type: z.string(), profile: z.string(), timeline: z.array(z.object({ year: z.string(), title: z.string(), description: z.string() })),});
const completion = await openai.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'You are a professional historian, capable of summarizing a person\'s life concisely and accurately.', }, { role: 'user', content: `Summarize the 10 most important events in ${name}'s life.`, }, ], response_format: zodResponseFormat(BiographySchema, 'biography'),});
// Expected output{ "name": "string", // name "gender": "string", // gender "type": "string", // type "profile": "string", // profile 100 words "timeline": [ { "year": "string", // year "title": "string", // title "description": "string", // description 50 words } ]}Caveats
Some OpenAI-SDK-compatible services don’t support zodResponseFormat
-
The deepseek-chat model doesn’t support
zodResponseFormatWorkaround:
response_format: model.includes('deepseek-chat') ? { type: 'json_object' } : zodResponseFormat(BiographySchema, 'biography'), -
The deepseek-reasoner model doesn’t support
json_objectWorkaround:
response_format: model.includes('deepseek-reasoner') ? { type: 'text' } : zodResponseFormat(BiographySchema, 'biography'),
Putting it together
const response_format = model.includes('deepseek') ? { type: model.includes('chat') ? 'json_object' : 'text', } : zodResponseFormat(BiographySchema, 'biography'),Summary
| Approach | Pros | Cons |
|---|---|---|
| Specify the format in the prompt | Simple, plain-language input | 1. Complex formats may take several rounds of tweaking 2. Different models interpret the format differently, which leads to malformed output 3. The output usually arrives wrapped in ```json ``` fences, so you have to extract the actual JSON yourself1 |
zod + zodResponseFormat | Clear schema, easy to maintain, supports complex types, highly controllable | Requires adding zod as a dependency |
Footnotes
-
Extraction:
text.match(/```json\s*([\s\S]*?)\s*```/)?.[1] || ''↩
LLM Structured Outputs
https://www.mihouo.com/en/posts/front/llm-structured-outputs/