Schema-guided reasoning vs structured outputs for LLMs
Use structured outputs when your program needs fields with a defined shape. Add schema-guided reasoning (SGR) when the program or a reviewer also needs intermediate records, such as source evidence and a proposed decision. SGR uses structured output to express those steps; it is not a separate guarantee that the answer is right.
If the consumer needs only a category and a source ID, keep those fields. Add analysis fields only when you can explain how they will be checked or used.
Last reviewed: 2026-09-08.
What each layer does
| Layer | What it provides | What it cannot establish |
|---|---|---|
| JSON formatting prompt | Instructions to return JSON | That the response follows a schema |
| Schema-constrained output | A response that follows the supported schema, when generation completes successfully | That the values are true |
| SGR schema | Named intermediate fields and a proposed decision | That one field was correctly derived from another |
| Application checks | Tests of source evidence, calculations, and allowed actions | Facts that the application never verifies |
Rinat Abdullin’s SGR description uses schemas as a checklist for the model’s returned work. In a self-hosted setup, vLLM structured outputs can enforce JSON Schema through a decoding backend such as XGrammar. Defining a Pydantic model in Python alone does not turn on constrained decoding in the server.
Check the selected provider’s supported schema subset and completion status. A refusal, truncated response, or failed request needs its own error path. The Gemini structured-output guide also draws the key limit: correct JSON structure does not guarantee correct field values.
A discount proposal shows the difference
Suppose a support tool may offer at most a 10% discount. A minimal structured reply could contain discount_percent and customer_message. If an internal reviewer needs evidence, an SGR reply might also contain policy_source_id and eligibility_evidence.
Those extra fields give the reviewer something to inspect. They do not make a proposed 15% discount valid. Even a schema that limits the percentage to 10 cannot prove that this customer qualifies.
Before using the offer, application code must load the applicable policy and customer record, check eligibility, and calculate the allowed amount. Treat the model’s explanation as a claim to check. If a later model step needs approved facts, validate them before making that call. Field order inside one JSON object does not create a validation step.
When the extra fields earn their place
SGR is useful when a wrong answer needs a more precise diagnosis. For example, a document classifier can return a source passage and a category. Reviewers can then distinguish a poor passage choice from a wrong category given good evidence.
It is less useful when every result gains a long explanation that nobody reads or scores. Extra output consumes tokens, and a convincing explanation can still be false.
Compare the small schema with the SGR schema on the same labeled cases and model. Score the final decision separately from source support and field validity. Keep the intermediate fields if they improve the decision or make review useful enough to justify the added output. Do not assume an accuracy gain from the schema’s name.
Deeper reading
- Schema-guided reasoning with vLLM explains schemas, XGrammar, and application-side policy checks.
- Schema-guided agent memory applies typed records and source history to persistent state.
- NER model choices separates literal text spans from extraction that requires inference.