There are probably some other axes where o3 is clearly better than o1, but with how complex intelligence is no metric will be perfect.
Also, is that graph assuming no changes in tokenizer if the measure is per-token?
Actually, trick question I suppose -- the base models already don't sound cringe, it's post-training that adds that!
(Well, some of them do, because a lot of companies are putting synthetic data and LLM outputs into the pretraining phase (as well as bot-infused post-2020 common crawl, I suppose), but the early base models were fully cringe-free.)
very noble of you, _Open_ AI
Clearly, if this should become an industry standard, public regulators must be able to audit both the datasets and how perplexity is calculated.