Top
Best
New

Posted by ankitvgupta 2 hours ago

Vibecoding an inference chip(ankitg.me)
55 points | 18 comments
cpldcpu 47 minutes ago|
I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

It seems implementing transformers is the new "hello world" of AI assisted coding.

datadrivenangel 1 hour ago||
Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.
risyachka 1 hour ago||
Fpga is software-driven, model can verify its output easily.

Zero surprise it did it

datadrivenangel 1 hour ago||
it only cost them $10,000 in Claude API usage. :)
hmate9 1 hour ago||
You get that much usage in a monthly $200 subscription.
dvh 47 minutes ago|||
So who's paying the cost?
nyc_data_geek 2 minutes ago|||
Retail and society as ANT shares get pulled into index and wealth funds
stronglikedan 6 minutes ago||||
Everyone else that doesn't use the entirety of the plan they pay for.
ralph84 14 minutes ago||||
API pricing vs. individual plan pricing is just price discrimination. They're both positive gross margin.
surgical_fire 7 minutes ago||
People believe in some wild stuff.
exe34 45 minutes ago|||
Corporate customers
dofm 1 hour ago|||
And a kitten.
codekansas 1 hour ago||
This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

chalkycrimp 56 minutes ago||
Good chance I'm not the only one thinking this: share the code
spwa4 6 minutes ago||
I must say I do see one application for this, that you can't really match reasonably: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

(unfortunately for a decent model it will also be price competitive with a big nvidia card)

saejox 47 minutes ago||
i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.
Archit3ch 14 minutes ago|
Now do silicon. ;)
IshKebab 9 minutes ago|
Did you miss this?

https://architectlabs.com/blog/frostbite