discord support can fix such issues

almost of their business is hosting Sol ultra fast or whatever for OpenAI to use internally

Especially since they still serve Codex-Spark, which is dogshit.

I have a self hosted Qwen 3.8 27B and I find it unbelievably cracked and dedicated. It's at least credibly attempted everything I've thrown at it. Just today I had it write a toy compiler with a JIT backend just to test out a concept, and that was with 4-bit quantization and 8-bit KV cache. Something has to be going wrong with your deployment.

Check sampling parameters and chat template, make sure you have adequate context window, turn reasoning effort down. It should be able to one shot a small app without intervention.

I run it locally at q4_k_xl on a r9700 with kv cache bf16 and while it thinks a lot, it’s still fast enough to do the task.

This model had its knowledge replaced with reasoning ability. The chain of thought what makes this reasoning effective.

So this is why you need to let it think and don’t quantize the kv cache.

I want a Qwen 3.8 27B hosted locally but I don't quite have the RAM for it. And, I don't want to buy the RAM until I prove I can use it.

Yesterday I did have success with Gemma-4-12b with 128k context. It fits in my RAM and it's relatively fast on my hardware.

I had to give it prompts that are quite a bit different from the way I use foundation models, but I did get it to work quite well. I feel like I could learn it's differences and get good at using it for real work.

Which quantization?

yeah - used sign in with google

No. You buy a minimum of $10 worth of credit, then use it at $1.49/M rate. There is no recurring charge.

There is a separate subscription based plan, which is sold out now.

They make a giant inference chip. Their inference service is basically just advertising for their core value prop: hardware.

The CEO was on Gradient Dissent a couple years ago: https://www.youtube.com/watch?v=qNXebAQ6igs

The wafer only has space for 44 gb of sram. If they offload ram they lose the speedup of having everything on 1 chip (the whole point of cerebras).

Mostly economics I'm sure

It is 15x more expensive. Openrouter usually charges like 1/4 for cached input.

Most of the cost for agentic coding is input tokens, you pay for the whole context at each tool call or message. Output tokens is just a small rate