Maybe I'm too jaded, I expect all this nonsense. It's human beings doing all this, after all. We ain't the most mature crowd...

Another well written article on HN. Well done. Very much enjoyed the read. And I learned a lot.

Very funny to re-title this to something less critical.

Point 1. could just as easily be explained by all of the labs being very close, and wanting to jump ship to one that is closer, or that gives you a better deal.

> Right before “making tons of money to redistribute to all of humanity through AGI,” there’s another step, which is making tons of money.

I've got some bad news for the author if they think AGI will be used to benefit all of humanity instead of the handful of billionaires that will control it.

Since no one has any idea of how to achieve AGI or the process to get there, I'm skeptical of any claims as to how soon we might arrive.

> A disturbing amount of effort goes into making AI tools engaging rather than useful or productive. I don't think this is an intentional design decision.

I think it absolutely is intentional. The overt flattery of LLMs is designed to keep you coming back because everyone wants to hear how smart they are.

The primary use case for AI-in-the-box is a superhuman CEO that sees everything and makes no mistakes. As an investor you can be sure that your money are multiplying at the highest rate possible. However as a self-serving investor you also want your CEO to side-step any laws and ethics that stand in your way, unless ignoring those laws will bring more trouble than profit. All that while maintaining a facade of selfless philanthropist for the public. For a reasonable price, your AI CEO will be fine-tuned to serve your goals perfectly.

Remember that fine-tuning a well-behaved AI to do something as simple as writing malware in C++ makes widespread changes in the AI and turns it into a monstrosity. There was an HN post about this recently: fine-tuning an aligned model produces broadly misaligned results. So what do you think will happen when our AI CEO gets fine-tuned to prioritize shareholder interests over public interests?

Im reading the "AI"-industry as a totally different bet- not so much, as a "AGI" is coming bet of many companies, but a "climate change collapse" is coming and we want to continue to be in business, even if our workers stay at home/flee or die, the infrastructure partially collapses and our central office burns to the ground-bet. In that regard, even the "AI" we have today, makes total sense as a insurance policy.