The demand for fast, cheap, good enough models has always been borderline infinite, it’s the supply that’s going to take off.

I forked my Big Serious Harness™ that models construction projects into a harness for building a vibe coded family assistant. I couldn't figure out how to make the toy operate at toy prices until Luna. Now you can vibe code all the little apps you might want for your fam for like $5 and operate it day to day for a few cents.

> Peter runs multiple companies. Beyond Segment, he's raised $100m+ for Charm Industrial, and just recently closed a Series A for Revoy. He's incredibly organized and efficient with his time.

You can do this before an exit? Build and fundraise for multiple (3?) companies at the same time?

Watch reasoning tokens though. We tried a small reasoning model that burned ~2800 thinking tokens per call, 3x the cost of a cheaper non-reasoning one despite a better price sheet.

I think we'll see more of this soon

replit is already leading the way with free luna usage

using small local models - with a little bit of extra work - for the first time over the past few days was _really_ illuminating and inspired similar thoughts about how far you can practically get with so little. column of zap emojis, mane ...

The "good enough" concept is interesting because of how systematically people over estimate it. So often, things that are lower quality but thought to be "good enough" turn out to be either not good enough or not worth it compared to just using the higher quality "thing".

I will believe that smaller models have hit that bar empirically when I see them in production. At the moment, even frontier models are stuck in most of the scenarios I am seeing for high value tasks at the "not good enough" gate - so small models are not even close to being on the scene there yet.

That's why the market has our solutions but economics.

I recently had some relevant experience: for a couple of months now I've been experimenting with on device models to summarize feeds in a Bluesky client I am developing. The feature extracts topic areas, categorizes posts, and creates a summary under each topic.

At first the results were hot garbage, and progress was slow. I hooked up the settings to download models from Hugging Face conveniently, so I could run experiments faster, and I massaged the prompts a bit. Last week this feature made a qualitative jump from science experiment to something I'd actually use.

The fact that all runs on the device means I've got no variable costs associated with adding this to what will be, at best, a pretty low revenue product. I've tested it on trailing edge devices like an M1 Mac and a Pixel 8, and performance is very tolerable.

The key is I'm not asking for open ended answers to open ended problems. When it proves to be useful it's not going to get less useful or more expensive.

There are vast domains of uses for LLM models with similar characteristics and likely similar results.

you both deserve a hot place in the HELL