Top
Best
New

Posted by acossta 4 hours ago

Perplexity and Intelligence(openai.com)
347 points | 229 commentspage 4
WASDx 1 hour ago|
Do they say that they measure perplexity at the pretraining phase? I thought most of the advancements have come from RL during posttraining. Would an older model with modern posttraining have the same perplexity while being significantly better? Then I don't see it as a meaningful metric.
reasonableklout 1 hour ago||
A common view is that RL is eliciting behaviors learned in pretraining [1], so even though many recent gains are achieved via posttraining, pretraining is still extremely important because base models provide all the latent potential. This is discussed a bit in the OP.

[1]: https://www.interconnects.ai/p/elicitation-theory-of-post-tr...

famouswaffles 1 hour ago||
They are measuring the perplexity of pre-trains.
Imnimo 2 hours ago||
>The primary held-out datasets we’ve been using since GPT-3.5 are sourced from our internal codebase, and those are the numbers reported here (for a fixed snapshot)

Am I right in understanding this to mean that the code on which we're computing perplexity was all written pre-3.5, and so it almost certainly entirely human-written? In other words, this isn't a snapshot of today's internal code base which contains a great deal of AI-authored code.

deadlypointer 1 hour ago||
Kind of expected, they are increasing number of parameters, and the model can fit the data better. Does not mean anything about "emergent capabilities" in my reading.
yewenjie 2 hours ago||
I am having a really hard time imagining what qualitative difference would gpt-5 and gpt-6 have in predicting the next token? What kind of dataset is it? How big is it?
XenophileJKO 2 hours ago||
This feels fundamentally wrong. As we move up the intelligence arc, I would expect perplexity to increase in general and only decrease on the pre train.

I would expect the perplexity on the original pre train to increase as reinforcement learning on verifiable domains teaches the model lessons that did not exist in the pre train. In addition alignment and other concerns, unless the are consistently enforced in pre train data quality will also move perplexity up.

athrowaway3z 2 hours ago||
This feels fundamentally right to me.

The best compression you're going to achieve is when you best understand(i.e. encode) the truth of things.

Perhaps you're thinking of that too narrowly - it is not that the model gets good at understanding how a bridge stays upright; it has to also understand the truth of how a specific person might understand it, and how that specific person might put it into words.

Do that right, as slight deviation of some understanding of how humans respond in general instead of encoding for a single person, etc, etc, etc - and you reach optimal compression.

It's more efficient for the model to encode _how_ an idea is wrong, than it is to encode the wrong idea.

Its how/why they detect when they're being evaluated.

SsgMshdPotatoes 2 hours ago|||
I think the perplexity is measured on the pre-train checkpoints.
stevenhuang 1 hour ago||
To effectively pretend to be a fool, the model needs to be smarter than the fool. That's why perplexity decreases.

The meta game in pretraining is it correctly assumes the context/mind state of the writer of whatever material under test. To do this effectively requires further modeling and understanding, which is a strong predictor of capability and is a much more difficult task than simply being bad at it.

rdedev 3 hours ago||
From gpt 5 haven't the actual harness played an important role? Pretty sure no model is a cyber security expert by just thinking hard enough. I feel like this article is doing a reverse causation here

For example consider this: would it be possible to build a model with such a low perplexity score but it be pretty bad at cyber security tasks? What if we removed all cyber security related data and RLVR from its training?

The article presents an observation but treats it as causation

SwellJoe 2 hours ago||
The harness is almost irrelevant (you can prove that by running the same tasks in a minimalist harness with only a few tools and minimal prompt vs. a complex harness and long prompt like Claude Code or Codex). Often the minimal harness/prompt actually performs tasks better, faster, cheaper than the big harness. This includes on cyber security tasks.

But, RL improvements are certainly a big part of recent improvements for certain tasks. That's what got the models doing long-running coding and math and research tasks, without constant human prodding.

adrithmetiqa 2 hours ago||
Completely agree. None of these models are even close to experts in any field (without human prodding).
criemen 3 hours ago||
Here I was under the assumption that since GPT-4ish we mostly improved models via RLing, and that pre-training stayed (roughly) constant. Mind blown :)
amluto 3 hours ago||
> The primary held-out datasets we’ve been using since GPT-3.5 are sourced from our internal codebase, and those are the numbers reported here (for a fixed snapshot).

Why would OpenAI expect any meaningful relationship between their models’ ability to predict the next token of supposedly held-out chunks of their internal codebase and the overall capabilities of the models? I expect both to improve, but that’s about the extent of it.

tedsanders 3 hours ago|
I work at OpenAI. The correlation between perplexity and the quality isn’t 1, but neither is it 0. It makes sense that as a model gets better at learning pretraining data, it also gets better at generically learning whatever you’re trying to teach it afterwards. The relationship is imperfect and empirical, and if it breaks down in the future, we’ll of course adapt to different measures instead of dogmatically clinging to the old paradigm. But so far, it’s still informative.
famouswaffles 2 hours ago||
I think his point (unless i've misunderstood) is moreso if a codebase is really the best held out dataset for such a evaluation meant to be wide-sweeping. Does this plot look any or much different if the dataset was more encompassing to include other sorts of text ?
lejalv 1 hour ago||
I would like to see perplexity vs. model size
AlexCoventry 3 hours ago|
I wonder if someone could back out estimates of their model sizes, using this graph and scaling laws.
More comments...