They worked fine for me. Keep working at it until results are positive instead of rabbit holing into a failure mode with a blog post.

It's usually more productive to right about how LLMs work rather than how they don't. In this case especially, there's improvements that can be made to the schema, without forfeiting on the idea of schemas altogether

"CoT x JSON means you can't get JSON" is 2024.

Every model has built-in segmentation between reasoning/CoT + JSON.

I would like to see a real example, the one given is assuming wanting float and assigning int.

What if you put “float” instead of int to get the required number?

Also the post is missing another use case, enums in structured data. I’ve been using it successfully for a few months now and it’s doing a fantastic job.

If this analysis is sound, I wonder if it can be mitigated by using tools instead of structured outputs.

Doesn't the Claude APIs recently introduced ability to combine extended thinking with structured outputs overcome this issue? You get the unconstrained(ish) generation in the extended thinking blocks and then structured formatting informed by that thinking in the final output.

I've wondered if it's because structured outputs rely on visual cues to impart meaning, and turning them into token streams looses that spatial structure.

This seems less like a failure of structured outputs and more like expecting LLMs to behave like deterministic parsers — or am I missing something?

or tell it to output the data at the end as markdown and then do a second pass with a cheaper model to build the structured output

also, xml works much better than json, all the model guides say this

Interesting read and perspective. I had very good results with structured outputs, both text, images and tool calling. Also a lot of SDKs are using it, including Vercel AI SDK.

Thanks for sharing

I didn't have reliable structured outputs until I switched to Gemini 2.5