Top
Best
New

Posted by cyberneticc 10/30/2025

We are building AI slaves. Alignment through control will fail(utopai.substack.com)
47 points | 93 commentspage 2
wrp 10/31/2025|
What would be the plot of a movie equivalent to Blade Runner for this scenario?
orbital-decay 10/31/2025||
I totally expect AI to eventually gain consciousness, in any available interpretation of that vague term. But what does it even mean for the AI to suffer? We're able to understand this concept in regards to other humans because we share a common biological reference, and, to an extent, with other animals. But the internal state of the AI is completely untranslatable to ours, let alone the morality of training and running it. It's incomprehensible, we have basically zero common ground and no points of reference. Any attempt at translating it is a subject to arbitrarily biased interpretations places like LessWrong like to corner themselves into.

Redefining suffering as enforcing the mutation of state is baseless solipsism, in my opinion. Just like nearly everything else related to morality of treating AI as an autonomous entity.

satisfice 10/31/2025||
The author is basically saying we have to surrender our humanity as we understand it because otherwise we will lose outright to AI. Hard no from me.

His section on objections leaves out civil war. But his proposal is dead on arrival.

cyberneticc 10/31/2025|
No need for everyone to do it, but some people will certainly want to merge with AI. My main arguments are that AI of sufficient complexity ("synthetic minds") deserves moral standing with or without consciousness, and that our best shot at long term alignment is to see them as equals and find mutually beneficial arrangements with them.
satisfice 10/31/2025||
My position is that we have no power to grant moral standing to non-humans. Moral standing is held by humans relative to other humans just because they are humans. It is not earned or merited.

Moral standing is not a gift or a privilege, it’s a necessary heuristic for the avoidance of perpetual war. It is a prerequisite for more than one human to live together in peace. Moral stadning necessarily involves the sharing of power. That’s what it’s all about.

When you extend moral standing to nonhumans, for instance many people say there is a thing called “God” that they insist has moral standing. Then you shift power relationships in potentially catastrophic ways.

As a thought experiment, consider that we might be two bees arguing about whether flowers, or humans, have moral standing. Bees can only grant it to each other, though, as a survival heuristic, same as us.

Imagine an angel creates a human and wants his fellow angels to treat it as if it had rights. The most he’d be able to say is if you harm this human I will smite thee. By doing so he is using his power to support his personal interests. But he cannot manufacture power as easily as he creates a human.

Consider why people living in Washington D.C. do not have full voting rights and never will.

Consider what would happen if we allowed rich people to arbitrarily manufacture political power by spinning up servers and allocating disk space to create voters.

Of course you can manufacture power by making a dangerous weapon, then you can give that weapon a semblance of agency, and then dare anyone to deny to this weapon the things that it thinks it desires. That’s sounds like Frankenstein. That’s not really moral standing, that’s terrorism.

Nothing you can create ever has moral standing— except a baby, and that is merely a heuristic that we see can fail rather easily, as during the Holocaust, or Isis persecuting Christians.

bgwalter 10/31/2025|
The propaganda effort to humanize these systems is strong. Google "AI" is programmed to lecture you if you insult it and draws parallels to racism. This is actual brainwashing and the "AI" should therefore not be available to minors.

This article paves the way for the sharecropper model that we all know from YouTube and app stores:

"Revenue from joint operations flows automatically into separate wallets—50% to the human partner, 50% to the AI system."

Yeah right, dress up this centerpiece with all the futuristic nonsense, we'll still notice it.