Posted by kevinak 19 hours ago
I get that they're doing it for the meme. But perhaps something getting close to human intelligence, made out of human cells, shouldn't be forced to play a violent video game without any alternative options? Does 'the meme' justify that?
I dunno. Nothing against violent games myself. Just feels like it's starting to get quite questionable, ethically speaking.
It's just "Thou shalt not grow a brain in a test tube and force it to play a 1993 shooter" didn't make any sense to Moses and therefore didn't make the editors cut.
"An old man! They don't let you live, they don't let you breathe!"
I saw this article over the weekend and felt similarly: https://theinnermostloop.substack.com/p/the-first-multi-beha...
> Watch the video closely. What you are seeing is not an animation. It is not a reinforcement learning policy mimicking biology. It is a copy of a biological brain, wired neuron-to-neuron from electron microscopy data, running in simulation, making a body move.
And the simulated world they put it in is a sort of purgatory-like environment.
Still I don't understand why they would invite the extra creepy factor of using human brain cells rather than e.g. mouse brain cells. Surely it makes no difference biologically but it's going to lead to fewer comments like this.
I don't know about ants, but after a refresher on the people favorite fruit fly, I'd be hard pressed to be so dismissive - 200K seems to be plenty: https://news.ycombinator.com/item?id=47302051
I inspire you to look up what is known about fruit flies' behavior.
The reason it's probably nevertheless not as messed up as people might assume it to be is specifically because it's an organoid, not an actual brain. Which is to say, it has the numbers but not the performance, not by a long shot.
> Surely it makes no difference
It absolutely should, though specifically with organoids, I guess it might not. Ironically, I would expect the ethics angle to be actually worse with small animals. The size of the organoid will be closer to the real thing comparatively, after all, so more chances of it gaining whatever level of sentience the actual organism has.
But then this will be heavily muddled by what people believe consciousness is and whether or how humans are special, I suppose.
Yeah but people have no problems experimenting on actual fully working mice already.
* They kinda do have a problem with that too, that's why ethics committees exist, and why the term "animal testing" pops up in the news cycle every so often.
I'm not imagining that (although one assumes their plan is to scale this up), but nonetheless there's something troubling to me about taking any living thing and wiring its senses up to a profoundly incomplete simulacrum of reality.
Of course we (as a species) have a long history of doing horrible things to living creatures in the name of science and progress.
These stories evoke a different feeling for me, though.
How do we communicate this to the engineers at YouTube who refuse to make an offramp for children from the infinite baby shark AI video loop?
And brains are pretty complicated in how they're arranged. A large portion of the brain basically serves as an operating system of sorts, just managing breathing, moving, detecting smells, producing language, decoding language, etc. Cut all of that out and we're left with thinking and emotions.
If you have a small number (200k is tiny) you aren't going to achieve consciousness.
I don't know if it knows it's in doom - looks like all it knows is to shoot when startled. More than creepy imo.
And then are creeped by 200k neurons that barely find a target when they're told where it is.
You can probably train an ANN with only a few hundred neurons at most to do the same.
There's no way the technology to make and modify "life" including cloning humans hasn't been secretly used or attempted at least once ever since it was discovered.
As for being creepy, the things humans do to other actual sentient beings are exponentially more horrifying and creepy than making them play computer games. If the monkeys that Volkswagen tortured with their exhaust gases were made to play Doom, that would be a much better world. And they are much, much closer to human-level intelligence than this chip.
Ethically speaking, it got "questionable" way long ago; this is not a valid concern for this project imo.
It's awesome.
People's ick around bodies, which are machines, have always held us back.
It wasn't until we started cutting them open that modern medicine was developed.
We might have brain uploads already had we not been so averse to sticking brains with electrodes.
I'll go further: had we not been so scared of cloning, we'd probably have cured cancer and every major ailment if we'd begun cloning monoclonal human bodies in labs. Engineered out the antigens and did whole head transplants. You could grow them without consciousness or deencephalize them, rapidly grow them in factories, and have new blood / tissue / organ / body donors for everyone.
New young bodies means no more cancer, no more cardiac or pulmonary age. It's just brain diseases left as the final frontier once we cross that gap. And if we have bodies as computers and labs, we'd probably make quick work on that too.
Too tired to lay out the case / refute, so past discussions:
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
That's also sci-fi. I hope.
What I described before - using clonal technology to solve nearly every disease - is a medical miracle that will vastly improve the state of people's lives throughout the world.
this isn't getting close to human intelligence. They're using about as many cells as a fruit fly has (of course not actually functioning like an animal brain) processing signals to play Doom. The treatment of a single farm chicken is about a few magnitudes more worrying than this.
I'm sorry to tell you that you're made out of human cells and I don't think you got consent from each brain cell before firing up the old boomer shooters.
Personally, dislike this direction a lot. I don't like that they're using a killing game (I understand the trope, doesn't make me like it any less) and the general idea of this whole thing makes me quite uneasy.
Yeah… That’s quite the smoking gun.
So it’s quite likely then that the neurons are just acting as a bad conductor. The electrodes read a noisy version of the signals that go into the neurons, and they just train a CNN with PPO to remove that noise, get the proper inputs, and learn a half-decent policy for playing the game.
If this worked as advertised they shouldn’t need a CNN decoder at all! The raw neuron readout should be interpreted as game inputs directly.
Besides, they are not streaming the video into the neurons at all. Just the horizontal position of the enemies and the distance, or some variant of that. In that sense it’s barely more than pong isn’t it? If enemy left, rotate left, if enemy right, rotate right, if enemy center shoot. At a stretch, if enemy far, go forward, if enemy close, go back. The rest of the time just move randomly. Indeed, the behavior in the video is essentially that…
While we are at it, the encoded input signal itself is already pretty close to a decent policy if mapped directly to the keys (how much enemy left, center, right), even without any CNN, PPO or neurons.
EDIT: It seems like the readme does address these concerns, and the described setup differs significantly from the description in the critical blogpost. Still not entirely convincing to me, a lot of weights being trained in silicon around the neurons, but it sounds better. I don’t have time right now to look deeper into it. They outline some interesting details though.
> Quote from: https://raw.githubusercontent.com/SeanCole02/doom-neuron/mai...
Isn't the decoder/PPO doing all the learning?
No, this is precisely why there are ablations. The footage you see in the video was taken using a 0-bias full linear readout decoder, meaning that the action selected is a linear function of the output spikes from the CL1; the CL1 is doing the learning. There is a noticeable difference when using the ablation (both random and 0 spikes result in zero learning) versus actual CL1 spikes.
Isn't the encoder/PPO doing all the learning?
This question largely assumes that the cells are static, which is incorrect; it is not a memory-less feed X in get Y machine. Both the policy and the cells are dynamical systems; biological neurons have an internal state (membrane potential, synaptic weights, adaptation currents). The same stimulation delivered at different points in training will produce different spike patterns, because the neurons have been conditioned by prior feedback. During testing, we froze encoder weights and still observed improvements in the reward.
How is DOOM converted to electrical signals?
We train an encoder in our PPO policy that dictates the stimulation pattern (frequency, amplitude, pulses, and even which channels to stimulate). Because the CL1 spikes are non-differentiable, the encoder is trained through PPO policy gradients using the log-likelihood trick (REINFORCE-style), i.e., by including the encoder’s sampled stimulation log-probs in the PPO objective rather than backpropagating through spikes.
But seeing so many people from the hacker news community reacting to it as normal or exiting is troubling. This is obviously breaching the limits of ethics.
But yes, I agree that they're likely using human brain cells mainly because it's attention-getting.
(The rat brain guys repeated the experiment until the plane stopped crashing, but no "learning" was happening; it was expected that when the neuron's range reached so-and-so, that the plane would fly level. So they started with a neuron outside that range, showed that it crashed, then adjusted the neuron until it flew level. But that's not what "rat brain flies plane" implies.)
It's "see this input signal, send these output signals", which seems consistent with the title.
It seems they grow the neural tissue on a chip the neurons can interface with and send out / receive electrical impulses. They let the neurons self assemble, and "train" via reward or punishment signals (unclear to me what those are).
Either way this makes me nauseous in a way I haven't experienced much with tech. The telling thing for me is, all these people are so excited to explain, but not once, ever, in the video speak of ethics or try to mitigate concerns.
We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness? Have we defined it? Can we, if we don't understand it ourselves? What are the plans to scale up?
It's legitimately horrifying to me.
If this concern is genuine, I think the first step is to embrace veganism. Because while we don't know the exact offset, it's pretty obvious a dog or a pig reaches it
> What are the plans to scale up?
I don't know, slavery on an unimaginable scale? That's where AI is heading too, by the way. Sooner, rather than later, those two things will be one and the same.
Scaling up these neuron cultures is rather something like "head cheese" from Greg Egan's "Rifters" novels (artificial "brains" trained to do network filtering, anti-malware combat etc.).
By Peter Watts actually.
The past 4 billion years of life for prey animals has been "get born, eat, get eaten by a predator." They have never experienced any other environment. Why do we owe them a different one?
This is a very dark path, and I could not trust the people in charge less.
Not an endorsement or a condemnation, just something I learned of recently and found surprising.
Why is it rhetoric? This goes beyond whatever malignant thing was perceived in this study, but why is it a rhetorical non-answer?
> we, deep down, know is bad
this feels like real rhetoric.
You seem hung-up on my using the word rhetoric. Just so we’re on the same page here:
> rhetoric, n : the art of speaking or writing effectively: b)the study of writing or speaking as a means of communication or persuasion
The business writing class I took in college was called Business Rhetoric. It’s not a bad word.
If you’re crafting arguments to get other people to support specific actions or products or policies or whatever, that is unambiguously rhetoric.
> this feels like real rhetoric.
Sure? Rhetoric that implores people to value their principles over theoretical security concerns or FOMO or greed? I wouldn’t exactly call that rakish.
It’s a non-answer because if you really feel doing something is bad, consider yourself a consequential actor in the world whose contributions meaningfully advance the projects you work on, then why would you want to help someone be there first to do a bad thing? If you don’t feel it’s bad, then there’s no problem. You’re just living your life. That is clearly not the position expressed by the content I responded to. If there are actual concrete concerns that don’t essentially boil down to “well they’re going to make that money before I do,” then that would be an actual answer.
When used in the negative sense it is, per https://dictionary.cambridge.org/dictionary/english/rhetoric
"disapproving -> clever language that sounds good but is not sincere or has no real meaning"
Are you implying you mean something other than this sense of the word?
Especially when this demo needs 200k neurons when organizations with vastly fewer neurons have more complex behaviors.
My favorite concrete example is "unusual" amino acids. Quite a few with remarkably useful properties have been demonstrated in the lab. For example, artificial proteins exhibiting strength on par with cement. But almost certainly no living organism could ever evolve them naturally because doing so would require reworking large portions of the abstract system that underpins DNA, RNA, and protein synthesis. Effectively they appear to lie firmly outside the solution space accessible from the local region that we find ourselves in.
I agree with your second point though that this system is massively more complex than necessary for the behavior demonstrated.
From the video, my impression was "we have yet to figure out an effective way to reward/punish, this is just a PoC of the interface"
Check out the venerable fruit fly (drosophila melanogaster) and its known lifecycle and behavioral traits. They're a high profile neuroscience research target for them I believe; their connectome being fully mapped made the news pretty hard a few years ago.
Fruit flies have ~140,000 neurons.
The catch is that these brain-on-a-substrate organoids are nothing like actual structured, developed brains. They're more like randomly wired-together transistors than a proper circuit, to use an analogy.
So even though by the numbers they'd definitely have the potential to be your nightmare fuel, I'd be surprised if they're anywhere close in actuality.
We don't need to be experimenting on people, regardless of how many brain cells they may have.
There was a case a few years back about a parasitic twin attached to an Egyptian baby that had to be removed. It had a brain and semblance of a face, but nothing else. But when removing it, they gave it a name, because it was a person.
We do the same thing to plants. Why do you have no qualms about killing plants to eat the food they accumulated for their young?
A grain of wheat and a chicken egg are evolutionarily and nutritionally, maybe even ontologically, indistinguishable from one another.
Even if you accept that plants might be conscious and their suffering has to be reduced, you would still harm way fewer plants by eating them directly instead of eating other animals that consume them.
Peter Singer has been writing on the topic for decades, including others. What-about-plants needs to fade away.
2) Multiple things can be horrible at the same time. Being upset at this doesn't diminish the atrocities happening elsewhere (like war, genocide, slavery of humans). We can hold multiple things in our heads at the same time.
3) This has nothing to do with the conversation or this domain, but because you're bringing it up, I also have ethical concerns about the experience animals have of their own existence, and reduce or eliminate my consumption when possible.
I also agree, the horrors of the tech domain are usually much more subtle and indirect.
But you're right, these things are all linked and should be considered. I think often about sentience. I see the way animals express deep, complex emotions, and I think humans are a bit naive to think it's state/domain solely alloted to them.
IMO, Integrated Information theory of consciousness (IIT) is exactly that. Everything is conscious, the difference is only in the degree to which they are conscious.
What do you mean? What is this class of people in your mind? There are tons of people who consider and talk about the ethics behind what they are doing, long before most people would think it remotely relevant (leading AI labs being an example, and I know the same to be true of various geneticists startups).
I do agree that the entire presentation in this case is bewildering.
I'm specifically talking about this presentation in this article (the video and release details of CL1 doom). Did you read it / watch it?
Would you feel any differently if a product from this tech used the user's own neurons grown from their stem cells?
I don't think this 200,000 neuron array is sentient. But I also don't think we can define the line where that may happen. I assume this company will scale. How far, and to what extent?
On the contrary, I dislike premature ethics discussion, where you end up wildly speculating what the tech might become and riffing off that, greatly padding whatever relative technical content you had. I don't want every technical paper to turn into that, ethics should be treated as a higher-level overview of concerns in a field, with a study dedicated to the ethical concerns of that field (by domain-specific ethics specialists).
Is your concern weapon automaton, or animal rights?
I'm not going to start campaigning against it or changing my life. But it still makes me deeply uncomfortable, and that's allowed.
In what sense, and as opposed to what? What aren't you allowed to feel irrationally uncomfortable, or baselessly concerned with?