Qwen 3.8 Max and Opus 4.8 score highest.

>with settings, tools, and context

call me crazy, but I think that this kind of suite, which training-uber-alles people generally dismiss as trivially replicable ‘wrapper’ is actually the differentiating factor for LLM adoption today and moreso into the future.

I’m not dismissing the near-all-out impact of training, but from a competitive busines or industry-structure lens, we’re looking at the three or four big players competing their utmost ultimately, if unintentionally, to turn foundation model access into commodity.

To the capabilities-maximalist minded (typical among engineers - my former life so I’m familiar don’t lack guilt in committing that) folks who will say “Oh the foundation model megacorps will just build out any wrapper whenever one of their third party wrapper plays demonstrates enough adoption, my rejoinder:

Apple did not rebuild an Uber-like app and cut Uber out.

We’ll see how this OpenAI legal services industry wrapper plays out, but I suspect 1) the third party legal wrapper plays will run to other foundation models not doing a legal wrapper, and 2) 3rd party wrappers will do a better job of it since it’s their all-out focus, unlike OpenAI’s whose priorities are necessarily more generalist.

Yes, we all remember the breakout startup failure-arguing quote “Google has entered your space.” That worked for several high profile applications. I believe more of those bets died on the vine than broke-out succeeded however, we only remember the biggest ones that persisted.

If legal services AI turns into one of the Mail or Maps-scale applications of the AI industry, while that would be a fair strategic action counter to the thesis I’ve laid out, the thesis itself would still tolerate it. It’s a question of short-fat tail vs mid-to-long-tail application scope & attractiveness. For example, I think it’s clear that coding is one of these short-fat-tail applications, and the low-no code plays are absolutely having their lunch eaten to acqui-hire ‘death’. I just doubt that the same will persistently transpire facing all professional service wrapper plays.