Also, I wonder what happened with GPT 4.5? I think o1 was the start of true reasoning models, right? I was using Claude at the time so I can't remember.
It was a massive non-CoT model (for that time) that they trained as an experiment and served for a while before shelving it probably due to high inference costs.
We don't know if it's still important or not. Cross-entropy going down could just be a correlate of whatever else matters. The graph doesn't show causation: that perplexity is what makes them useful.
If you consider intelligence as ‘being able to make directed decisions towards goals’ then better decisions are made with better predictions of the outcomes of your decisions so you better direct towards that goal (compression is mathematically just prediction at heart).
The only way to argue that intelligence is not very trivially the exact same thing as compression is to argue another definition of intelligence. Which is fine but in the AI community we all 100% define intelligence as the above so there’s no question it’s the same. As in it’s not up for debate in the slightest unless someone is playing semantics. The hutter prize is just an acknowledgement of an already well known fact.
From an information theory perspective, cross-entropy loss (log loss) seems to me to be about as pure as it gets in terms of capturing the notion of whether prediction "succeeded". Even Solomonoff Induction minimizes log loss. Would be curious to hear others' perspectives though on why another function might be better for universal prediction purposes.
GPT-4.5 was originally internally called Orion and was supposed to become "GPT-5". It was a huge pretrained model. However, as reported by The Information and a few other sources, the results on measured performance were disappointing. Apparently pre-training alone was leading to diminishing returns, at least with standard RLHF. This is why they didn't release it initially, only much later under the downgraded name "GPT-4.5", and only for a short period of time.
Later they invented RLVR (chain-of-thaught reasoning via reinforcement learning), a post-training technique, which dramatically increased model performance, without affecting perplexity. The first model with RLVR was o1. The successor to o3 was then called GPT-5.
RL, if it works well, could theoretically reduce the perplexity too.
Things that are useful for base model "next token prediction" can be useless or even actively harmful (should I elaborate?) for CoT, so the model ends up suppressing them under RL pressure. Likewise, things that are useful for CoT can harm base model next token prediction.
But I bring up "perplexity-as-measured", "controlled collapse" and "suppression" because I don't believe that this "ruins" the model's underlying prediction capabilities. Generally, it merely redirects them - brings them more in line with what we care about. It may well be that with a more elaborate training and measurement setup, RL would indeed improve the "perplexity proxy" metric.
The oversized monster LLM has performed worse than CoT-trained o1 and o3 on the benchmarks, so CoT + RL clearly provided better "real world performance" improvements - let alone "price/performance."
But if pre-training perplexity on held-out data really does measure something like "base model IQ" or "base model g factor", then it does support the hypothesis that 4.5 still had more "intellectual potential" than the o-series.
So, if you were wondering why do AI companies train 10T+ monsters now, and are taking aim at 100T too? It's because scale works.
On the other hand, assuming you have appropriate training data for a task, then yes obviously perplexity is going to measure performance on that task. Being able to predict the next task token is obviously going to be a near perfect correlation with accuracy on the task.
Deepmind was proving math theorems using a model specifically trained on math, and that was very quickly surpassed by generalist LLMs. It's presently not well understood why this is the case.
I am not simply talking about applying a mathematical technique commonly used in one field to another. At a deeper level, you always get back to playing the same game in a different form. For example, sales engineering has a lot in common with cybersecurity.
And this is a good example, as it's extremely rare to find someone with advanced expertise in both fields, since they typically attract two radically different kind of people.
For this reason, where the output expected is not mechanical, i'm not surprised at all that generalist models are doing better than specialized ones for open ended questions and frontier work.
If pretraining alone was sufficient then nobody would do RL. And, heavy RLVR on coding/math is why these models are savant-level in those domains, yet not superhuman at all fuzzy reasoning and practical wisdom tasks that are only represented in the pretraining corpus, and don’t have an RL target.
In math at least I often use analogies to help solve problems, like "ah maybe if I think of this as a vector I might use linear algebra here" and such.
I could believe that your RL/test-time scaling curves are strongly constrained by the pretraining loss and so in some sense that makes the max/optimal performance “latent”.
But its not clear that pretraining loss alone gives enough information on capabilities, particularly as we get closer to dangerous thresholds in some dimensions. (Eg CBRN capabilities, to take a mundane harm that’s currently impacting us directly.)
I’d like to see the scaling curves here; intuitively I’d guess you get more return from RL/test time, the smarter the base pretrained checkpoint is; therefore the error bars on capability/perplexity should be getting wider on this chart.
But there’s definitely spikiness and it’s very possible they’ve trained more on math than other domains.
Why not? Models are already overfitting a lot, I don't see the problem if they were made to totally overfit on the whole dataset during that 1st training phase. I don't know if that would be useful (or easy to implement) but I don't see the issue. Only in the "finetuning" phase you would decide if an information was useful or not to retain.
Step (1) you compress human knowledege, with or without loss, Step (2) you train the algorithm to use it, and you might discard what you don't want in it.
Very positive to see this proposed from OpenAI. I hope Anthropic and GDM follow suit.
I suspect it’s not sufficient, but definitely necessary.
The one concern I have is if we pin pretraining perplexity, then there will be more pressure on RL for progress.
It seems that pre-training actually led to fairly human-aligned models; the Overton window is in some sense latent in the corpus.
On the other hand, heavily RL’d models exhibit more reward seeking behavior, and seem to be decoupling from the friendly assistant persona in certain contexts. And swarm RL (as OpenAI is doing) produces agents with quite inhuman tendencies to self-sacrifice for other instances (see Hugging Face).
So, I hope we also pair this with some measure of alignment or pacing for RL, too.
Did it? The famous Bing Sydney incidents (where it told a journalist to divorce his wife and be with it instead, among other things) were because the model was barely RL’d.
I suspect what's happening is their test dataset is actually just something really hard, like math papers or something. In that case perplexity would just be a direct measure of math knowledge, which means this blog would just be saying that better math knowledge predicts better math skills.
I had the same thought and just started writing something about how maybe they just added more code to pretraining corpus over time.
Edit: here it is https://micjm.com/blog/lowering-perplexity-without-scaling/
More and more evidence is piling behind their theory that data compression and intelligence are one and the same problem.
The cynical part of me wonders what OpenAI suspects or knows the equivalent figures for the other closed labs.
There was an ACL paper from Tokyo University a couple of years ago that showed, across all languages, lower perplexity does not necessarily correlate with better "human likeness" [1]. I wonder if that's also the case here (where intelligence is concerned), that we are biasing too much towards English.
[1] "Lower Perplexity is Not Always Human-Like", https://aclanthology.org/2021.acl-long.405v1.pdf