Top
Best
New

Posted by neomindryan 1 day ago

Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU(www.neomindlabs.com)
318 points | 205 commentspage 3
rvz 1 day ago|
...and how many servings can this do?
neomindryan 1 day ago||
Author here. The short version: a viral post ran Gemma 4 on a 2016 Xeon; my Xeons are 2013, and the fork it used assumes AVX2, which Ivy Bridge doesn't have. The build failure was easy. The fun bug was the silent one: two MoE graph ops with no dispatch case on non-AVX2 builds, so every expert FFN output was uninitialized memory. Deterministic, NaN-free, fluent-looking multilingual gibberish.

The fix is open upstream as PR #2138 (https://github.com/ikawrakow/ik_llama.cpp/pull/2138), awaiting review. Fair warning on the AI angle: the patch was written by Claude at my direction. The post is explicit about which parts were me and which weren't. Happy to answer questions about either the bug or the workflow.

why_only_15 22 hours ago||
Why is the post AI-generated? If you're going to make something for us to read you pay us the courtesy of actually writing it first.

https://www.pangram.com/history/61fbc90e-180d-4d91-a85d-16ff...

otherjason 1 day ago||
This reads as pretty clearly AI-generated text, which is against HN guidelines.
FL410 1 day ago|||
The PR? He said it was AI in the comment you replied to...

I don't think the post itself reads like AI at all, but that's just me.

why_only_15 22 hours ago|||
The post itself is totally AI-generated. It has tons of tells, in addition to Pangram saying so.

https://www.pangram.com/history/61fbc90e-180d-4d91-a85d-16ff...

twoodfin 1 day ago||||
The post is absolutely LLM-generated. “Punchy” short sentences, “… has quietly come to mean …”, “The optimized paths weren’t there to execute.”
logicallee 1 day ago|||
I think "this" refers to its parent comment. Part of it sounds like Claude wrote it. AI-generated comments aren't allowed on HN.
otherjason 1 day ago||
Indeed, I was referring to the parent comment.
pkghost 1 day ago|||
Here's the thing: life also imitates art. If you invert your load-bearing assumption, it could be that he just reads too much slop. But my honest take? You might be right.
dofm 1 day ago||
stopitgetsomehelp.gif ;-)
linncharm 22 hours ago||
[flagged]
vugar82 15 hours ago||
[dead]
0x457 1 day ago||
[dead]
OsamaJaber 1 day ago||
[dead]
CurbStomper 1 day ago||
[dead]
rvba 1 day ago||
Apologies for asking here but literally nobody knows:

Android studio connected to a local model disconnects automatically after 10 minutes. How set this limit to 12 hours or remove it completely?

I could run my LM studio model all night... but I cant, since Android studio times out after a hard limit of 10M.

This is not related to number of tokens.

I tried Googling, searching for settings in Android studio, even created a stackoverflow post - but zero information. Jetbrains mentions "remote agent timeout mechanism" - but after changing it, nothing happens.

NortySpock 1 day ago|
If the local model is served via ollama, there's a default timeout of 10 minutes , which can be adjusted either per-call , or (as I did) in the systemd service environment variables

https://docs.ollama.com/faq#how-do-i-keep-a-model-loaded-in-...

You didn't specify what was serving your local model.

rvba 1 day ago||
Thank you for your reply. I use LM studio (local server), but can switch to a different tool.

Do you know how to switch it in LM studio?

What I see is that: android studio gives "Error: stream failed" and in LM studio server I see it is still working, then says that client (=android studio) disconnected.

So I assumed it was a setting in android studio.

NortySpock 1 day ago||
Dunno, I have not used either of those. (Had been using zed and ollama, and ollama had plenty of odd defaults that needed fixing)

Glancing through the docs, I would be digging down in the config of both Android studio and lm studio for either a TTL or jit auto evict setting, and if you find it, set it to some large number measured in hours?

https://developer.android.com/studio/gemini/use-a-local-mode...

https://lmstudio.ai/docs/developer/core/ttl-and-auto-evict

rvba 6 hours ago||
I saw those and sadly none of those works not provides solution to "Error: stream failed" that happens like clockwork.

I found this one:

https://www.jetbrains.com/help/toolbox-app/remote-agent-time...

But it requires me to reinstall android studio because jetbrains toolbox cannot find it... because I installed ir on D:

On a side note Android studio atill creates hundred of megabutea of hidden stuff on C:

indigodaddy 1 day ago|
Unfortunately, the post comes off as AI-written. Why not just write your own posts?
CookieCrisp 1 day ago|
Not everyone is as confident in their writing as they are in their engineering