The first time your pipeline runs clean, the JSON validates, and the schema checker gives its silent nod of approval, it is tempting to believe the hard part is over. Valid JSON is the baseline, not the finish line. The five failure modes it names, the kinds of errors that survive constrained decoding, live in the gap between syntactic correctness and semantic truth. A model can hand you perfectly formed keys and values while quietly mislabeling a date, swapping a customer ID, or inventing a plausible-sounding number that never existed in the source. Your validator will never flinch. It was never designed to.
This is the moment where the conversation about structured outputs needs to grow up. For years, the promise of LLM structured outputs was framed as a battle against malformed JSON, and tools that guarantee valid syntax felt like a victory. But anyone who has actually deployed these systems in production knows the harder problem is trust. A schema confirms structure, not substance. It cannot tell you that the extracted entity is the right one, that the classification is mutually exclusive, or that the model did not hallucinate a reasonable-looking value because it was easier than admitting uncertainty. Constrained decoding solves a mechanical problem, while the failure modes that matter most are semantic. That is not a criticism of the technology; it is a challenge to how we evaluate it. If you are building on top of these models, your testing strategy has to include human review, adversarial examples, and a healthy respect for the fact that a valid response can still be wrong in ways that are invisible to your entire toolchain.
What would we tell a reader who asks whether this means structured outputs are not worth using? The answer is not to abandon them, but to stop treating them as a safety net. Structured outputs are an efficiency gain, not a correctness guarantee. They save you from the embarrassment of unparseable responses, but they do not save you from the quieter, more dangerous failure of confident nonsense. The practical shift here is in how you design your evaluation pipeline. You need to test for semantic accuracy, not just syntactic validity. That means building datasets where you know the ground truth, running your extraction pipelines against them, and measuring how often the model produces a valid but wrong answer. This is harder than running a linter, but it is the only way to know if your system is actually working. The burden of proof has moved from "did it parse?" to "did it understand?"
The specific detail worth watching is the fifth failure mode, the one that no amount of schema design can preempt. It is the class of errors that emerges from the model's own training dynamics, the confident misreading, the subtle conflation, the plausible invention. No validator will catch it because the problem is not in the format; it is in the model's relationship to the truth. The takeaway you should quote: "Valid JSON is the price of admission, but it is not the product." If you are building with LLMs, your real quality bar is semantic fidelity, and no schema checker can measure that for you. The next time your pipeline reports success, ask yourself what it is not telling you. The answer is usually the most important part.