Posted by lairv 22 hours ago
Why even rank Fable 5.2's capability in Cybersecurity if you won't allow us to use it for that?
> Mythos 5.2's safeguards stopped the response above
Opus 4.6 was the last useful model for my work. I plan to cancel at the end of this month. To replace it I'll likely go to either tinfoil.sh or privatemode.ai.
This means that they are blocking the research of AI to different people other than their own?
https://artificialanalysis.ai/
GPT-6.1 Sol costs $1.27 per task, whereas Fable 5.2 is $26.54 per task -- a ~21x price difference.
Fable 5.2 is not 21 times better than GPT 6.1 Sol, I find it hard to justify this.
6.1 sol is also an exceptionally cheap model in your comparision. On 6.1 sol high is $0.59, which is actually the same price as Deepseek 4.1.
That's for coding at least. For creating images (SVG, 3D models, etc), medium does usually suck, and every level above improves it.
How is this relevant? I'm willing to pay 21x more as long as the cost is less than my value of time saved
AI generally underperforms human, so is "raw quality" is the only metric you would not use an AI. Time-to-completion-at-quality is usually pretty important and Fable is not good there.
It also burned through my entire 5-hour limit in about 10 minutes. Granted, it had almost finished the whole review in those 10 minutes, so points for speed I suppose.
It's weird they've left that as a "gotcha", and I can't think of many charitable reasons for doing so.
I'm pretty confident (from internal evals and monitoring over nearly six months now; across many models) API has never been affected by A/B tests / "nerfs" / etc.
But FWIW for Fable specifically, I've noticed this via API rates from Anthropic directly. Eg: added cash to my $30/month account to use Fable, and still get obvious nerfing.
https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
max is in there too, but it ran out of reasoning tokens (128,000 limit) before it got to the actual SVG.
Interesting how low detail these were all up until xhigh.
High gets legs correctly, but screws up the layering on the chain.
x-high is very good but there are minor laying error with the front chainring.
https://jonclegg.github.io/pacman-bakeoff/#claude-fable-5-2
Then I clicked on it and realized it refused to make a Pac-Man game, and instead made up a new game. That's actually very slick.
If they are already overreaching here, and needlessly, what if their AI was actually super intelligent? Where will Anthropic stop?
"Make it more realistic"
Well, the result is something. Be aware that it took 20 minutes and used 3 percent of my weekly Fable token (Ultra plan):
https://claude.ai/public/artifacts/c7e842c0-bf92-41af-82c6-3...
I know the latest Gemini release explicitly had something like this.
My experience working with Opus medium over Sonnet xhigh is similar, it's similar quality but cheaper and faster.
I hate to make a comparison with humans but I guess my view is you can get a better answer more quickly by asking a more intelligent human too.
gpt-6-luna XHIGH as the reviewer agent still found plenty of bugs that Fable 5.2 missed!
Here was Fable's summary of the session:
The reviewer ran 16 reviews over 9 commits this session (7 rejections, 9 required fixes, all valid); three of them, on the plan, changed how the numbers are computed.