Posted by teleforce 4 days ago
The performance of local AI for coding, given correct setup & harness, is very good and on par for 95% of use cases compared to closed models.
Local usage for end-users is unmetered and token speeds are very fast and keep improving every few weeks due to optimizations being done in the community.
Data remains local which makes EU compliance checkboxes way easier since data never touches anything related to the CLOUD act.
While I don't know your specific use cases, when interpreted generally, this seems trivially disproven by putting eg. websites made by Opus 5.5 and the local model side by side.
I am currently building an integrated CAD + Aero + CAM software package for rapid UAV design iteration, and the open models are all not very good at it. They make a ton of mistakes and I have to end up redoing a lot. And that's using very detailed specs and TDDs provided as context to the models. Opus/Astra/Argon/Gemini Flash 3.8/Sol does a significantly better job in my experience.
What's the problem?
Frontend has been solved for years.
With GPT-5.4 or GPT-5.5, or of course also by writing the code by hand if we want to go all the way, it was well possible to build nice frontends to our liking.
I still don't want to go back when I can do it better and faster with Opus 5.5.
I am geninuely interested what stack you are using here. Could you please elaborate?
It feels like things changed in the past few months. We are now truly able to do what we need to do without Claude etc, where this was not feasible earlier this year.
Who is "we"?
We're back to early 90s prices.
EDIT: Yes, obviously you don't need a RTX5090 to play games. But it seems to me like the top GPU market is now exclusively for the local LLM users.
I have 30 years of coding experience. That tiny model is no Opus 5.5, but make no mistake: it's a very strong coder in the hands of a capable developer.
It's harder and harder to justify gaming PCs having separate pools of GPU-only memory and "everything else" RAM.
I think unified memory is the obvious next step for personal computing and vendors will be coming up with solutions that enable more modular approach to building enthusiast grade upgradable personal computers.
And if a system is intended to be modular and upgradable, what would combining the GPU and CPU into a single module contribute to this goal?
Now an architecture that separates the memory controller from the CPU and gives pluggable CPUs and GPUs (and storage and network controllers, for that matter) equal access to a single pool of RAM in a secure and high-performance way, that would certainly be interesting, and not only or even primarily for gaming.
I imagine the PCs for the future would maybe have motherboard with two sockets: a normal CPU one and a GPU one, both having access to some high bandwidth unified memory that.
Right now the unified memory needs to be pretty much soldered super close to the chip to ensure the bandwidth and energy efficiency, but hopefully there will be some solution to circumvent this limitation in the future.
Putting CPU and GPU cores into the same physical die works for your average consumer machines, but hardcore professionals or enthusiasts would always strive for more custom configurations than existing packages can provide, like more GPU cores.
Integrated systems have come a long way for highly optimized mobile and laptops designs with specific hardware in mind. But even if everything went perfectly - DLSS 5 on a 5090 halves the frame rate and requires a 5090. And it can only do a single style - gritty, high contrast lighting photorealism.
In 10 years will new GPUs primarily be AIPUs? Sure, probably, but they'll still be discrete and they'll still have significant raster and ray tracing pipelines to provide the basis of rendering.
Then again, maybe AGI will come and hardware will fundamentally revolutionize overnight.
The technology is very new and currently in its infancy. I do believe it's the future of computer graphics though, where instead of doing "real" photorealistic geometry and lighting, software can render a very simplified scene and have a good enough photo-realistic filter applied to it so that most people wouldn't be able to tell the difference.
Obviously, I believe it'll be flexible enough to do things other than photo-realism in the future.
> In 10 years will new GPUs primarily be AIPUs? Sure, probably, but they'll still be discrete and they'll still have significant raster and ray tracing pipelines to provide the basis of rendering.
I think we have different definitions of "discrete" here. In my mind AIPUs are the opposite of discrete. I might be wrong here, but I always interpreted this term as a standalone GPU unit with its own onboard memory — a thing that currently fits into a PCIE slot.