> > one NVIDIA GB200 NVL72 That’s around ~$4-6 million in hardware cost (one rack, 72 GPUs + accompanying hardware).

that’s a good clarification, skimming some of the comments here I was thinking they trained it on a single GB200 gpu. THAT would be a huge win for indie AI

Edit: Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems. The claim of “we trained this on a single rack of gpus” is definitely much less impressive than “we trained this on a single gpu”

> they didn’t have to worry about hosting/billing/subscriptions or any of that yet

Or marketing, which they kinda get for free. So much research goes unseen, there’s huge competition for attention, and most research is very limited in that regard

I would say that’s the biggest factor that makes the team “not indie”

At the same time, like you said, still amazing accomplishment, very cool model, and great work

Also a speech to speech model it can't do any tool use or you can't guide it much am I mistaken? How will it be used for agentic uses is the part I don't get. Edit: this is response from Argon if anyone was also curious: You are not mistaken about Dessa: because it was trained solely on raw audio with zero text data, it cannot read text system prompts or output code for tool calls on its own. To use it as an agent, developers must either steer it purely through spoken voice instructions or pair it as a conversational "voice front-end" alongside a separate text LLM that listens to the audio and executes the tools in the background.

Wait, seriously, a rack of 72 GPUs costs $4 million?

Perhaps boutique is a better term?

What else did you think all the Fallen Empires in Stellaris did ? ;-)

Some say they’re still in there to this day.

Also with people having fewer ties to the rest of humanity, there will probably be some weird effects on morality

Some will, and they will outcompete.

human advancement is a function of population, instead of focusing on population decline to solve our resource problems we should focus on efficiency and technology. i am not rooting to revert to the dark ages i really struggle with this ideology of reduction that seems extremely popular. we really take for granted the fragility of human existence we exist in a pico second of reality and could and most likely will be snuffed out eventually, having our own species rooting for this outcome seems suicidal.

so pretty much anyone who isn't amish?

In the future you won't be able to tell.

Maybe you just have a voice that's, shall we say, in sparse latent space? ;)

I have an insane amount of writing output as training data going back a loooong time so it all comes down to how much training data do you have

Guessing - finetuned? Or LoRA?

EDIT: Nevermind, see below. I would definitely go with a LoRA though.

Couple things:

I have a “style guide” markdown that is pretty expansive

I run an RLHF style loop with me in the middle for all my writing now where I give a voice rant that is over expressive (aka redundant samples of concepts that I want to express clearly) in order to manage lossy compression steps

Then I will manually clean up that last 5% after a half dozen rounds

That sequence goes into the style guide to be referenced as a reasoning chain of thought to map rant > output

It’s been three years of tuning I’d suspect at this point