Posted by adocomplete 8 hours ago
Gets wrong some tests. It does answer correctly, BUT it doesn't respect the request to respond ONLY with the answer, it keeps adding extra explanations at the end.
Interesting. I wonder what the exact question was, and I wonder how Grok would respond to it.
```
/model claude-sonnet-4-6[1m]
⎿ API error: 429 {"type":"error","error": {"type":"rate_limit_error","message":"Extra usage is required for long context requests."},"request_id":"[redacted]"}
```
i cant believe that havent updated their code yet to be able to handle the 1M context on subscription auth
https://web.archive.org/web/20260217180019/https://www-cdn.a...
My bets are its more the increased hardware demand that they don't want to deal with currently.
i.e given an actual document, 1M tokens long. Can you ask it some question that relies on attending to 2 different parts of the context, and getting a good repsonse?
I remember folks had problems like this with Gemini. I would be curious to see how Sonnet 4.6 stands up to it.
Opus 3.5 was scrapped even though Sonnet 3.5 and Haiku 3.5 were released.
Not to mention Sonnet 3.7 (while Opus was still on version 3)
Shameless source: https://sajarin.com/blog/modeltree/
(Sonnet is far, far better at this kind of task than Opus is, in my experience.)