Almost as if this stuff has extremely little practical use outside of niche categories.
Ideally the hold out set is randomly sampled. You need a hold out set of Inklings pretraining run.
Should we trust that the dataset of each company is big and diverse enough to be representative? I don’t hope too much…
ln(5,7) = 1,74
did you even use the right logarithm? because they use the natural logarithm..
edit: that puts it before GPT-3.5 even
And it again feels equally dishonest/bullshitty.
You can't really make an objective statement that coding requires multiple orders of magnitude more intelligence than having a conversation. And that solving math problems requires orders of magnitude more intelligence than coding.
It's just marketing talk. In reality all three are separate skills and the real intelligence requirements are, if you ask me, not that far away from each other.
To put this in an analogy, it feels like charting "athleticity on a log scale" and putting first running a marathon, then swimming across English channel, then benchpressing 200kg, each one presented as requiring an orther of magnitude being more athletic.
Where are you getting 'orders of magnitude'? and yeah I do think it's reasonable to say that Useful Programming requires more intelligence than the median human, and Solving millenium level math problems certainly requires considerably more intelligence than the median Programmer. What exactly is so absurd or unreasonable about that?
In the same way you could say that beating Gary Kasparov at chess requires more intelligence than the average human and yet we had an AI winter.
Essentially, Humans who tend to have higher verbal skills also tend to have higher spatial skills, better memories, and faster processing speed. Someone with the cognitive ability to become an exceptional chemist might just as easily have become an exceptional mathematician or software engineer (all else being equal).
The knowledge and skills required are obviously different, but the underlying cognitive abilities that make someone successful in one intellectually demanding field often transfer to others.