Posted by Brajeshwar 1 day ago
The rule of all games played in capitalism is there's a sucker born every minute. you can work a few years or your whole life before you realize you've been had.
and most do....
2. Humans put it in charge of everything, as fast as possible, because it's just so damn useful
3. Is it perfectly aligned to human values for all of future ????
4. Everybody dies as a byproduct of the ASI pursuing some project that humans don't even have a chance to understand, like an ant colony during construction of a hydroelectric dam.
If so, he's the person that's made me lament one loss the progressives took from us. I think there's one phrase that we should be allowed to say regardless of the fact that it is also used to refer to certain groups of people. Because the insult comes from the comparison, not the language itself. Every time we update the language, we just use the new word as the insult. And I don't think we're allowed to use the word here.
But sometimes, you just have to look at someone, cock your head to the side, and say "That's fucking ****".
It brooks no argument, it offers no sympathy, it just tells the other person that they should be ashamed for speaking those thoughts aloud.
I studied biology in college. I actually did this because I was interested in AI, and already knew a ton of CS and programming. I was already coding in C and using Linux and writing fairly complex stuff. I figured studying the only actual example of a truly intelligent system (especially at the time, early 2000s) was the best way to understand intelligence.
Didn't end up going into AI as my day job, but I've kept up with it all and worked through the math and played with different types (genetic, neural nets, transformers/LLMs, CA / alife, etc.).
I think the root of my skepticism is in what I learned studying biology about how life, adaptation, learning, and evolution actually work in the real world.
I'll try to TL;DR it: I think the doom crowd (and also some of the AI singularity crowd) has a "mystical" view of intelligence. Intelligence is magic pixie dust, and intelligence beyond human is fully unknown and likely has magic superpowers.
The deeper underlying thing seems to be some kind of mathematical mysticism.
Real intelligence is ubiquitous in nature. Trees, grass, bugs, and even bacteria are intelligent. It's also domain specific and limited in capability to what its domain, available training data, and available compute allows.
It's not context-free limit-free magic. It's not special. It's an adaptation like any other.
Both OpenAI and Anthropic are from that world, where Rationalist doesn’t mean you’re a rational human being, but instead that it is rational to believe that AGI will be created, thus that they should be the ones creating it. It’s a doomsday cult.
If you start looking at the links between prominent rationalists and effective altruists you see how that whole doom prophecy came about. It’s also how they got so much capital in the early days, SBF literally invested into Anthropic as part of his Effective Altruism effort foundation, and both Amodei siblings are deeply into that whole world
Once again, I've said it elsewhere here, I think we could have nipped this thing in the bud if when the first "effective altruist" started talking about the EV of saving billions of "future lives" vs dealing with today's real problems, someone looked at him, cocked their head to the side, and said "That's fucking ****".
All of those magical future people do not get to live if we bake the Earth today. The best way to take care of future people is to take care of people today. And if they were truly serious about taking care of future people, that's what they would be doing. But it's not, they're just collecting resources. I don't know how the fuck Sam Altman owning a Koenigsegg Regera today helps save 20 quintillion lives in 40,000AD. Deeply unserious people.
I still can't believe anyone ever took Roko's Basilisk seriously, but people did... to the point of freaking out about it.
The groups themselves are a little cultish too, or a lot cultish depending on which one you're talking about. Some of them are full-on cults.
But what do I know. I'm probably low IQ.
If you've spent much time in US Christian circles you've probably encountered people suffering from apocalyptic thinking. The Roko's Basilisk folks seem like the atheist version of the same crowd. It's hard to take them seriously when they're always seeing portents of the end times, and their thinking is clearly clouded by anxiety and fear and not based in reality.
Bacteria and up are full of gene regulatory networks that are absolutely intelligent and share some features with neural networks. Bacteria learn, cooperate, signal, react, seek goals, and that's in addition to evolutionary learning which is also fast for them given their reproduction rate.
Evolution in nature isn't just zeroth order either. Look into a topic known as "the evolution of evolvability." Evolution does meta-parameter search on itself, more or less, and invents adaptations to improve its ability to adapt. Sexual reproduction is the most significant and largest example but there are others. There's environmentally dependent tuning of mutation rates, genome self-editing that we still don't fully understand (this is also how our immune system works), lots of epigenetic stuff we are still learning about, and so on.
If it's alive it's intelligent. All life is constantly learning.
Interesting, before Yudkowski was patient zero for AI doom he was a nanotechnology "grey goo" doomer. That was even more obviously absurd to me. Grey goo already exists. It's called biology, and it's been trying to maximize its footprint on Earth for 4.5 billion years. The miraculous instant "foom" grey goo thing is magical and likely violates conservation laws.
& ofc AI is making climate change worse, some of the Amazon datacenters are using fossil fuels to generate a country’s worth of power, but that’s never part of their calculations
Which one of these would it be? All of them in parallel? Or something we wouldn't think of. If I were to play chess against Magnus Carlsen, I wouldn't know with which piece he will checkmate me, but I know that I am going to lose.
People do things.
Good or bad is relative to their own biases/experiences/ beliefs and moral compass.
Very few people are truly sociopaths who do evil stuff for personal gain.
Most do it because of the curse of the “banality of evil” (re: <https://en.wikipedia.org/wiki/Eichmann_in_Jerusalem?wprov=sf...>)
Doesn’t make it right, but like everything else in life the reality is much more complex than saying “good people refuse to do bad things”
My current read is that ~20-30% of people in America actively delight in hurting others (ICE supporters etc)
Another 30% do not seem to care about anything but their own well-being. (I don’t like to think I fall here but as a big tech peon it doesn’t look good).
The remaining percent seem to care and are trying their best but looking at how things are going it’s not enough
The lesson from history is that doing good is the most reliable path to do evil. To do actual good: leave other people alone, unless they ask for your help.
"AI safety" practitioners fit the mold of historical great-evil-doers well. They are the true danger in AI.
Obviously morality and ethics aren’t binary. OP isn’t saying that. What they’re saying in a clear and concise manner is that your actions have consequences far beyond your personal sphere, and good people acknowledge that and fight for what’s better despite the harm they personally may suffer as a result. That what makes someone “good” in their point of view isn’t a binary deed or outcome, but of the sacrifices they’ll make for themselves to build a better existence for others.
Planting a tree whose shade you’ll never enjoy, in other words.
So to reiterate OP’s challenge at the end, at least be honest about why you’re still in the game. Let others know where you stand so that we don’t waste our time on futility. If your voluntary position on something mutable risks making you a target, then the rational next step would be to evaluate how important that position is to you that you would weather said harm instead of changing it.
That’s all OP was getting at: promoting the mere notion of critically thinking about the decisions you make in regards to your own moral philosophy and acting more authentically, including tanking the damage trying to bring about change.