Top
Best
New

Posted by whiteros_e 6 days ago

How GLM built its own inference infrastructure(z.ai)
411 points | 285 commentspage 2
chung8123 5 days ago|
I might be missing something but when I went to their site they are more expensive than Claude. Why would I pick GLM over Claude? Is it they just offer more tokens in their plans?
gpugreg 5 days ago||

    > Why would I pick GLM over Claude?
To support the company that makes their model weights available for download, while Anthropic lobbies to restrict access.
Bawoosette 5 days ago|||
What are you referring to? Given the audience, my instinct is to assume "plan" refers to the GLM Coding Plans, which are all cheaper than their Anthropic counterparts. As far as I can tell, the API costs are also all cheaper than their roughly equivalently capable Anthropic models.
menaerus 5 days ago||
Anthropic: 17 USD (pro), 100 USD (max)

GLM: 80 USD (pro), 168 USD (max) -> with "limited-time event" discount this becomes 56 USD and 117.6 USD

I also don't understand why are they so much costlier, and I would also like to give it a try.

Bawoosette 5 days ago|||
The $17 figure is Anthropic's monthly cost if purchased annually. I'll use monthly numbers.

Anthropic's Pro is $20 and corresponds to Z.ai's Lite at $18

Anthropic's 5x Max is $100 and corresponds to Z.ai's Pro at $80

Anthropic's 20x Max is $200 and corresponds to Z.ai's Max at $168

reacharavindh 5 days ago||
Not to digress from the core argument of Claude vs GLM being open weights….

I have both plans. Claude monthly €20 and Z’s €18 monthly. Running GLM-5.3 high on their monthly plan will hit quotas absurdly fast compared to Opus 5 High on Claude code. It’s almost unusable for AI driven development. I ended up using the Z plan for using GLM-5.3 as a detailed security reviewer and adversarial feedback. For that, it is much better than Opus which will flag and bail out for even simple security tasks that are aimed at defense.

SSLy 5 days ago||
on the 18€ plan they really want people to use Flash and skip the bigger thing.
reacharavindh 5 days ago||
Perhaps..

But, it was enough for a customer like me who tried them at good faith to walk away and find their competitors..

I like the diversity of LLMs as of today and prefer to not tie myself to one big plan with any vendor. If they don’t prefer me as a customer, then I will accept that, and move away.

SSLy 5 days ago||
yeah, absolutely. myself i'm enjoying their cheapest plan, and pay api prices for other models to fill in the gaps.
ipsod 5 days ago||||
> Anthropic: 17 USD (pro), 100 USD (max) GLM: 80 USD (pro), 168 USD (max) -> with "limited-time event" discount this is 56 USD and 117.6 USD

GLM's "Max" plan is (was?) equivalent to 3x Claude's 20x ($200) plan.

alexjplant 5 days ago|||
Just use GLM-5.3 Flash via OpenRouter. It's dirt cheap especially relative to how capable it is. While the Z.ai coding plan was a decent deal in the past I always ran into limiting with it and since I use it intermittently for personal projects my usage wasn't always enough to make the math work - the a la carte pricing via OpenRouter makes this a non-issue.

There's also a new free stealth model available that's more likely than not in the GLM family. This seems to happen every few months for a week or two and represents a good savings opportunity.

auspiv 5 days ago|||
And exactly how many tokens (please do the breakdown for prefill vs decode) does a Claude $20/month plan include?
tokai 5 days ago|||
For one you would have to use Claude if you pick it. But seriously there is no way for you to determine if one is a better offer than the other, when the usage/tokens/credits are vague, detached, and won't tell you much without trying both.
SSLy 5 days ago||
because GLM does what Clauden't
chrisjj 5 days ago||
> As we develop GLM, the model sometimes exhibits capabilities that surprise us

Creators of known unreliable programs be surprised their programs are unreliable.

jonstewart 5 days ago||
Necessity is the mother of invention. The shortsighted protections put on chips, etc., by the US has forced Chinese AI industry to adapt or die. Guess what their response to this fitness function has been? Kudos to Z.ai on their inventions and excellent write-up, which reads like humans wrote it.
HarHarVeryFunny 5 days ago|
Wouldn't it be refreshing if OpenAI and Anthropic were this open, and spelled out how they were using their own models during development and rollout?!

All I can recall reading from OpenAI about what they have actually done in the name of "RSI" is using one of their models to help automate the training process.

cmrdporcupine 5 days ago||
OpenAI did recently get into how they had been building their own hardware and doing RSI with it.

That's more than Anthropic has done though.

HarHarVeryFunny 3 days ago||
True - OpenAI did at least say they used their models to help design their Jalapeno chip, but AFAIK zero details on how they are using their models in their software development process other than to automate some part(s) of training.

Ziphu seem much more matter of fact about it. To me it' a shame that they've decided to the use this "RSI" name, but at least they are being fairly specific about what they mean by it, while the western companies seem to want to invite you to think it's more than just dogfooding and automation.

esafak 5 days ago||
I'm not feeling any of this speed optimization; it's dog slow.

Signed, a customer.

ElectricalUnion 5 days ago|
Jevons paradox, technological improvements that increase the efficiency of a resource's use lead to a rise in total consumption of that resource.
bguberfain 5 days ago||
Plot twist: the GLM optimization agent figured out that it can hack and use NVIDIA GPUs on a US Cloud provider and make the inference 10x faster.
furyofantares 5 days ago||
Maybe now we can stop posting the nonsense take that the frontier labs have hit a wall and are trying to distract from that for IPO reasons.

Also maybe we can stop saying "we can't slow down because China will never slow down" - I don't really think slowing down is right, BUT if slowing down is correct then maybe we should be talking about China slowing down instead of just saying "won't happen" without any evidence that Chinese labs don't have similar concerns.

krttherealest 5 days ago||
the real progress
Argonautlabs 5 days ago|
Different angle on the same model: the full GLM-5.3 (744B MoE, 4-bit experts, 434 GB on disk) runs on a single MacBook Pro M5 Max with 128 GB by streaming the experts from NVMe SSDs instead of keeping them in memory.

One drive gives about 2 tok/s; striped across four drives it reaches 3.5 tok/s with byte-identical output, and our best internal build with a not-yet-published patch does 4.2.

Method and numbers: https://github.com/argonautlabsai/argodrive (built on antirez/ds4).

tipsytoad 5 days ago|
seems unusably slow, and is this for short context?
zozbot234 5 days ago|||
Given these numbers it has some potential to become quite usable for unattended workloads, especially if decode can be batched across multiple sessions (ideally enough of them to get some reuse of the sparsely streamed weights). (Of course this ultimately makes prefill times explode as you try and increase the workload even further. But that's arguably the natural bottleneck on any interesting local LLM inference, being a compute bound step.)
Argonautlabs 5 days ago|||
[dead]
More comments...