Top
Best
New

Posted by kschaul 2 days ago

AI 2040: Plan A(ai-2040.com)
377 points | 484 commentspage 8
0xbadcafebee 17 hours ago|
> In AI 2027, we predicted that this would result in either extinction or irreversible concentration of power.

Neither of these come to pass. The first because we don't live in a science fiction universe. The second because AI is a completely open source technology. You can literally make your own models and train them yourselves using state of the art techniques published in free papers. All you need is GPUs and smart people who can read (and now that AI can code, you don't need the second bit). These people are doomer quacks too caught up in fear and excitement to think rationally.

Their expectations and assumptions are all wrong, from the basic understanding of AI in general (like how you don't need a billion dollars or "American Brains" to make a half decent model), to the misunderstanding of market realities and competition in China and Europe. The US doesn't have a monopoly on chips, or on smart people; Huawei chips work fine for AI training and probably a fifth of the US tech sector's best workers aren't American to begin with. There's many forms of AI in many places; munition, economic driver, laborer, generic tool. It's now a part of the world, the way the Internet is. There's no keeping the genie in the bottle. AI doesn't take over the world; it's integrated with it. It's boring.

In a few years people will forget the doomer predictions, the way every doomer prediction that never came to pass is forgotten. We didn't nuke ourselves, the internet and television didn't rot our brains, the radio and telephone didn't corrupt the souls of Americans, the newspaper didn't incite anarchist riots. Every new technology freaks people out, and we adapt to every new technology and make it boring and normal. That's humanity.

adt 2 days ago||
Excellent work by Daniel and the authors. 47,000 words plus supplements is a huge read (and re-read), and an even bigger think-and-write.

My early analysis of the analysis:

https://lifearchitect.substack.com/p/the-memo-special-editio...

SubiculumCode 1 day ago||
Sounds like another Chinese Op to me; Ensuring Chinese compliance would be incredible hard to enforce or to check.

Look, I am scared of where we are heading, but I cannot see how we can change the dilemma towards mutual cooperation unless, as humans tend to do, only react massively after something really bad happens.

sometimelurker 23 hours ago||
> Ensuring Chinese compliance would be incredible hard to enforce or to check

consider SALT and other treaties from USSR vs USA cold war. they checked

supern0va 1 day ago||
I recommend actually reading their recommendation, because they get into the weeds about precisely how the US and China could address this in a trustless/auditable way. The TL;DR is that basically all of the relevant compute can be tracked.

Edit: Also, definitely not a Chinese op. The authors are prominent Americans, and are the folks responsible for the AI 2027 forecast that has pretty accurately predicted the current state of affairs today: https://ai-2027.com/

hkalbasi 23 hours ago|||
Pretty accurately what? So now OpenBrain has an Agent-1 that makes their algorithmic progress 50% faster than other companies? If it was 50% more CVEs, that would be something, but I doubt any meaningful self improvement is achieved which the competitors are slower due that, which is the core of the prediction for 2026.
SubiculumCode 1 day ago||||
I wonder. There has been some headway in getting decentralized training runs to work.
supern0va 1 day ago||
Yeah, Cognition's work is interesting in that regard, but it still doesn't obviate the need for the chips--it just enables training on them when they're spread across multiple data centers.

The Plan A proposal estimates that the ownership of ~96% of AI relevant compute hardware can have its ownership traced, since the companies selling are very few.

foobiekr 1 day ago|||
I doubt we can even track all of the chip production capacity.
sajithdilshan 17 hours ago|
I wish the authors of the article spent more time studying how LLM works before writing a doomsday scenario. Currently no matter how impressive LLMs are, they are just token auto complete machines. It cannot invent or think anything new which is not in its training dataset.
nonadhocproblem 17 hours ago||
Are you trolling? OpenAI's models resolved the Cycle Double Conjecture yesterday, which has been an open problem for 50 years.
sajithdilshan 15 hours ago||
Do you even know what Cycle Double Conjecture is? Or what even a conjecture means in general? It’s a perfect use case for autocomplete with existing knowledge
ainch 14 hours ago||
One of the lead authors, Daniel Kokotajlo, worked at OpenAI for years before quitting. In 2021 he wrote a remarkably accurate forecast of how LLMs would develop over the following 5 years[0].

I think it should be obvious that he understands that LLMs are trained via next-token prediction.

[0]: https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-...