Top
Best
New

Posted by apsec112 2 hours ago

Why I Left Google DeepMind(turntrout.com)
232 points | 123 comments
recitedropper 1 hour ago|
Nothing but respect to TurnTrout for taking an action like this. The world needs more smart people who are willing to stand for what they feel is right, despite the pressures otherwise. Without that occurring more, our species is going to lose many impactful prisoner's dilemmas coming these next two decades.

This raises my respect for AI researchers a little bit too. I have often felt that the entire industry is pretty tainted to the core, and for better or worse that colors my opinion of the researchers.

Maybe I'm in the minority, but I thought it was gross to download pirated art for a student project when I was at Berkeley years ago. So it has been really sad to witness many of the most brilliant minds of this generation answering the siren song of disrespecting the collective effort of others to extract and resell residual value.

I'd guess TurnTrout doesn't agree on that framing, otherwise he probably would not have been at Deep Mind. But clearly he and I agree on other ethical positions; I am nothing but glad to see him stick to his principles here.

yw3410 1 hour ago|
Apologies, but it would be good to add links for your anecdotes.

Not all of the readers of your comment have the appropriate context and know what you're talking about. I certainly don't.

recitedropper 1 hour ago||
Hmm, links for what? My student project? The decisions that face humanity that are pretty clearly modellable as prisoner's dilemmas?

Otherwise not sure what I could cite--I would assume most all on this forum know that AI is trained on the works of other people, without their permission to do so. I guess you could disagree with my framing, but I wouldn't think this requires a citation.

I think maybe my writing wasn't clear, and it sounded like I was referrencing some well known thing that happened at UC Berkeley. I have edited it to read more cleanly!

yw3410 1 hour ago||
Yes; the student project you were talking about.

It wasn't clear to me it was _your_ project nor who pirated it. I was under the impression it was a well known scandal from your original, unedited comment.

Apologies if my comment sounded hostile; I was just asking for a clarification/more information on it.

recitedropper 1 hour ago||
No worries! Thanks for engaging. Fun to have the rare kindly-resolved internet discussion.
slowin 1 hour ago||
Props to the author. I left Microsoft due to their work with Israel to spy on Palestinians and record all of their phone conversations (in addition to other IDF collaborations). They ended up walking some of it back, but Satya was complicit.

https://www.theguardian.com/world/2025/sep/25/microsoft-bloc...

za3faran 38 minutes ago||
I salute you for your morality. If most people did this, those companies would change their position very quickly. After all, they only think about money.
slowin 28 minutes ago||
Thank you but I'm no saint. I joined Microsoft knowing that they were incompatible with my values. I just told myself that I wouldn't work on those types of projects and try to stick to my values. I told myself that I could just be "professional" and keep my head down.

But, as I learned more I was filled with a deep sadness and shame. It was tearing me apart. Every day I got up to go to work and was getting sick to my stomach.

Now I just feel lost. I spent my life building a career in this industry and it feels like I have nowhere else to turn. Microsoft is not the only company actively complicit with horrible crimes and human rights violations. I've loved technology since as far back as I can remember, but looking back with the knowledge I have now, I just feel dirty. I honestly don't even know what to do at this point, I do have a job now but when I look at who funded us, it's every bit as bad as working at Microsoft. I feel like I need to walk away from it all and start a food truck or something.

h0mie 42 minutes ago||
Respect!
metalsiliconYT 1 hour ago||
Respect for sticking to your beliefs. Just out of curiosity though why do people not want smart AI weapons? I would much rather have an onboard AI that can discriminate between unarmed civilians and military assets, seems irresponsible to not... Is a dumb sea mine that blows up everything somehow better than a smart sea mine that knows to not blow up sometimes?
jodacola 44 minutes ago||
"You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"

I realize that's not a great argument and was definitely tongue-in-cheek, but given there's still a lot of debate about the accuracy of AI for far more mundane tasks, my personal perspective is that until we have LLMs and such that are truly, demonstrably far more accurate than humans, with true reasoning and judgement capabilities, they don't belong where lives are at stake.

I wouldn't want an LLM-underpinned machine running anesthesia during a surgery; why would I want an LLM-underpinned military apparatus that is deciding the lives of far more? I wouldn't, not in their current state.

In a hypothetical future where we truly trust incredibly smart AIs or LLMs or whatever "smart" technology it is for driving weaponry, okay - if it's truly necessary; I abhor war and the death and destruction wrought by it.

In my mind, though - even if we get to that future where there's some vastly superior technology to the LLMs we have today, which can judge and reason, then I'll have a bunch of other questions, like understanding the motivations of said technology, because I suspect it'll be something much closer to AGI, and that opens a whole separate can of philosophical worms.

tartoran 32 minutes ago|||
Not only that but an AI could follow any orders it is given (whether it hallucinates that's a different story) whereas humans may oppose. The friction of orders having to go through a chain of command through it's completion is a feature not a bug and that adds responsibility. AI can take no responsibility and with moving fast break things smart weapons that responsibility can be shoved under the rug.
logcode 37 minutes ago||||
> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"

What if it was the target though? AI may be more capable than we're giving it credit for (especially the AI accessible to the US and other governments). The attack on the girls' school coincided with Purim and I don't believe it was a mistake. I think it was the opening salvo by a radically religious Zionists (Christian and Jewish).

I don't think the biggest problem with AI weapons is that they make "mistakes", I think the biggest problem is that they allow people who want to kill civilians the ability to accurately do just that.

metalsiliconYT 38 minutes ago|||
So you would prefer weapons that cannot reason at all?

> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"

Most likely this event happened due to a bad targeting system that wasn't smart enough, if it had a better llm underpinning it (assuming it had any) maybe those lives would have been saved. More reason for more smart people like the author to work on these systems.

nvme0n1p1 8 minutes ago|||
> So you would prefer weapons that cannot reason at all?

Um, yes? It's bad enough humans are murdering each other. At least a human can in theory be held accountable for pulling the trigger. The last thing we need is an unaccountable ralph loop reasoning about which schools and churches to bomb every time it wakes up.

> if it had a better llm underpinning it

Ah yes, the "LLMs are intelligent, you're just not using the newest fanciest model" fallacy. This time with innocent lives on the line. If only we used ChatGPT-8.9 instead of 8.8, those poor kids would still be alive today.

Retz4o4 18 minutes ago|||
President Trump reveals the US Space Force has CAMERAS that can read the NAME TAG "Mohammad" trying to reach any nuclear dust in Iran

"We say: 'Mohammad something is there with shovels.'"

"We have cameras that can read the badge of the person. 'Mohammed Something.'"

"WE'RE WATCHING. If anybody goes there, THEY GET BLOWN UP."

"Eventually, we'll take it."

They have also admitted they saw the children’s flower chalk drawings too. And they double tapped.

cogman10 39 minutes ago|||
Mainly I don't want them because I don't want a machine instantly making life or death judgements. It might be fine if it mostly informs a human who ultimately pulls the trigger, but having that failsafe is important in keeping machines from going on killing sprees.

I also want people to be held accountable when they do unjustified killings. AI weapons make it FAR too easy to simply pass off a killing as a "woopsie doodle." It's just not acceptable to say "The algorithm made a mistake, version 23 will do better".

I don't have a problem with the AI providing additional information to it's user, but when that's incorporated into a weapon it's a short distance from that to completely automating the killing.

That's why I'm completely against AI weapons.

metalsiliconYT 32 minutes ago||
Responsibility will always have to lie with the user, but if the weapons are more advanced and safer then why is that bad? I like that autonomous vehicles like Waymo are 10x safer then human drivers, even if a 'Machine' is making decisions.
scarecrowbob 40 seconds ago|||
Because they aren't "safer" for the folks getting killed.

Historically, the folks doing the murdering just don't care about the folks they kill, so "safer for the killers" isn't a win for most of us.

"More precision" isn't about killing less folks, it's about making it safer and easier for folks who kill folks to do that work.

So those of us who dislike killing don't like these tools because we consider them to be immoral.

cogman10 10 minutes ago|||
> Responsibility will always have to lie with the user

Being fully autonomous makes it hard to identify exactly who that user is and is easy to dilute responsibility. Perhaps someone was added to a kill list by mistake. Maybe some internet hi-jinks tricked an AI into falsely identifying someone as a kill target. Perhaps it's the case that someone was in the 5% of a 95% confidence of identification. I'm not a fan of putting killing into the hands of something known to get things wrong 1 in 20 (95%), or 1 in 100 (99%), or 1 in 1000 (99.9%).

> but if the weapons are more advanced and safer then why is that bad?

It's yet to be proven that they are "safer" as they become more advanced. There's also a question of "safer to who". It's technically "safer" for a soldier to shoot first and ask questions later, it's obviously not safer for the villagers.

False identification, which is an absolute part of AI, doesn't make these weapons safer for anyone.

> I like that autonomous vehicles like Waymo are 10x safer then human drivers, even if a 'Machine' is making decisions.

Waymo has the reverse bias. If anyone dies as a result of waymo it's gone horribly wrong.

AI weapons are designed exactly to kill, if they don't kill when they should something has gone wrong.

Barrin92 42 minutes ago|||
>Just out of curiosity though why do people not want smart AI weapons?

When the decision to kill another human being is made that should be in the hands of a directly accountable other human being, not an unaccountable machine developed in the basement of a private corporation.

And mines, both dumb and smart, in particular anti-personell mines are banned by the Ottawa treaty ratified by 162 countries. It's exactly the autonomous and fundamentally uncontrollable nature of mines, not just that they're dumb, that has produced countless of casualties long after wars were over. Can you tell me that millions of autonomous loitering munitions are not going to end up exactly like those mines still blowing legs off people decades after conflicts are over? And who is responsible then?

metalsiliconYT 36 minutes ago||
The mines got banned because they were dumb...
righthand 37 minutes ago|||
They aren’t training smart AI weapons to discriminate between civilians and soldiers. Even so, what happens when the soldiers are disguised as civilians? Or when the enemy forces civilians to serve or be a decoy?
tgv 38 minutes ago||
Because it's going to be used in any possible way, and certainly not for the good of the people, as imply in your "naive" false dichotomy.
seydor 1 hour ago||
Principled people have become so rare
rvz 1 hour ago|
Only when the bank account reaches 7 - 8 figures.
chung8123 1 hour ago|||
Even at zero dollars and a billion dollars they feel rare. Honestly I am surprised how many tech people still work for Facebooks and Googles of the world when they actively have used their platforms against most of their values. These are skilled people that can still make a living elsewhere.
michelb 12 minutes ago||
I Think it’s because when you have enough money to never have to work again and you can move to some place nice, you can isolate yourself and your family from the rest of the world, like most rich people do.

So that’s what, 4-5 years at a society destroying tech co in exchange for lifelong freedom away from the people you have actively made life worse or impossible for.

It seems many people can live with that and in fact will jump at a chance to do it.

grim_io 1 hour ago|||
No need to downplay it. There are many for whom it's never enough.
devindotcom 1 hour ago||
And there are others who left years ago and warned us. Back-patting isn't all or nothing, and I think it's reasonable to praise those who left when they saw the writing on the wall more than those who leave only once they see their company materially supporting state killings.
grim_io 1 hour ago||
Fair.
0xWTF 40 minutes ago||
I haven't finished reading, but I should note for the group that the author doesn't represent the Anthropic situation accurately. Emil Michael, the undersecretary of war/defense for research and engineering went on the All-In podcast (1) and explained, for quite some time, exactly what happened. You should go listen to it (2). In essence, in the negotiations, Dario kept coming back saying "well, if you need it, call us, we can redline things as needed". This happened over and over and over. Emil's point was that a major conflict, if it were to occur, might happen on the 30 minute clock of an ICBM, and all due respect to Dario, the national security apparatus just doesn't have an allowance in that 30 minutes to call him for a redline. Emil felt Dario had demonstrated he wanted ultimate control via line item veto, and was willing to trade up to and including the survival of the nation for that veto. And a government cannot be expected to pay for that sort of behavior from an entity domiciled in their jurisdiction.

Now, Dario is going to win a decent slice of the economic pie. But as an military acquisitions matter, I gotta say, I have to agree with the undersecretary's position here, and yeah, it makes sense to document a company's undesirable behavior, and in certain circumstances push that information to others. Not the first time the government has found a company under contract acting in a way that appears to be counterproductive to the government's obligations; there's a whole database full of this stuff (3).

1) Without a doubt, All In is a friendly crowd for Emil, but I think that actually made it easier for him to get more nuanced facts out in this case because he wasn't spending a lot of time defending malignant attacks.

2) This link should jump to the relevant segment https://www.youtube.com/watch?v=gzwRflcLPAA&t=2479

3) http://www.ppirs.gov/

AnishLaddha 34 minutes ago||
That doesn't address the supply chain risk designation? Or the ban on all work with claude for contractors, incl. non-government work?

Even if they disagreed, this was clearly retaliatory for not capitulating. The government could have just decided to go with a different provider.

andyjohnson0 19 minutes ago|||
Michael is some marketing guy who used to work for Uber and now chose, voluntarily, to work for Trump - and gets to cosplay at being "Undersecretary for War". Why would anyone believe anything that such a person says?

And this "might happen on the 30 minute clock of an ICBM" is just another variation on the Ticking Timebomb "if you only knew what I know" forcing-function that gets deployed all the time by these brave warriors.

Analemma_ 34 minutes ago|||
I cannot think of a less trustworthy source of information than Emil Michael on the All-In podcast. Why on Earth would you take his statements at face value? I grant that nobody in this situation is totally trustworthy, but if it's his word against Dario's I'm believing Dario 10 times out of 10, and it's astonishing and dismaying that anyone might do otherwise.

Keep in mind that Emil Michael's job at Uber was to slander and destroy their political enemies. What makes you think he hasn't continued exactly as before?

PunchTornado 34 minutes ago||
Wait. Do you trust what he said?
dr_um 26 minutes ago||
Well done, great effort! Unfortunately, most people are weak in their comfort.
khalic 1 hour ago||
This is inspiring, thank you to the author, it must have a hard piece to write.
srameshc 1 hour ago||
I did not know about TurnTrout till now and thanks to HN community now I know. I am beyond impressed as to how he lives by his principles. I don't think I have this kind of courgae or confidence and thankfully I don't work for Google or have to quit it. On the other hand we have smart people who have choose to work for companies like Palantir and are proud of it.
Theodores 25 minutes ago|
It is not about courage or confidence, it is more a calling, where few are called and that often turns out to be just one person, or so it seems, since everyone finds their excuse to not take a stand against the oppressor and for the oppressed.

The author will have experienced every logical fallacy and TurnTrout has done well to document his efforts, rather than just the crimes of his adversaries.

A calling of conscience is hard to articulate initially, it is conscientious objection, to the war machine and the system of finance that necessitates the empire that needs the war machine.

The author starts off with a high degree of authority in that he actually worked for Google Deepmind, however, nobody will listen to him, so he has to seek higher authority to carry the truth forward. But he could have gone to the pope, anyone short of Jesus Christ (freshly teleported back) and it will be a no, everywhere.

Logic and reason does not help when insanity and money have taken over, which tends to happen during wartime.

The conscientious objector may not believe in god, however, they will consider themselves answerable, 'vain' enough to care about their legacy. There is that desire not to be found out, doing the 'devil's work', generally in workplaces where everyone is glad to get on with the 'devil's work'.

Why do so few people make a stand? Why didn't hordes of five-eyes people walk out the day after Snowden did his good deeds?

It varies by individual, however, the author is vegan, which means he has already 'dared to be Daniel, dared to stand alone', albeit only in a lunch queue in a meat eating world. This requires living according to principles, and serves as target practice for war-time conscientious objection.

Also important is a certain level of independence, the guy with a mortgage and a couple of kids, underwater on the car, with maxed out credit cards cannot conscientiously object. Nor can the guy counting down the last few years to retirement, and then the very young lack the articulation.

Only a few have the 'warchest' to embark on an open-ended single-person campaign to confront power with the truth. We owe a lot to these people, particularly the 'failed whistleblowers' that don't make it to 'whistleblower' status, because the media then makes the story about their situation, not what they conscientiously objected to. Props and respect to the author for documenting his journey and taking the first step.

Note the 'first step' is classic 'hero's journey', where the call is initially rejected, but then a journey into the extraordinary world is made, where the challenge is to bring back what is good from the extraordinary world to the ordinary world, for the benefit of all.

kombine 25 minutes ago||
This is no longer on the front page of HN despite 200+ points and having been posted 1 hour ago.
Chance-Device 11 minutes ago||
There’s a flame war detector, this has certainly triggered it and pushed the thread down the rankings.
SirHackalot 19 minutes ago||
I saw it on the front page.
Zsfe510asG 1 hour ago|
We know now that Google is a sycophantic company whose DEI initiatives were all fake and dropped once Trump got elected.

This somewhat naive initiative was bound to fail. The good news is that the AI military products won't work, except perhaps for blowing up a girls' school.

Here are CEOs falling over themselves to support Hassabis' regulatory capture proposal:

https://xcancel.com/sundarpichai/status/2077086951833063580#...

https://xcancel.com/satyanadella/status/2077063479232795024#...

https://xcancel.com/elonmusk/status/2077415601610297535#m

It is an exclusive club and we are not part of it.

metalsiliconYT 1 hour ago|
Saying AI military products wont work is the dumbest thing i've heard in awhile
Zsfe510asG 54 minutes ago||
Search for Anduril drone failures in Ukraine and several Lattice software failures. And perhaps capitalize "I".

How is Maven working in Iran?

metalsiliconYT 51 minutes ago||
Thats like saying cars wont work because they get flat tires
windexh8er 30 minutes ago||
It's almost like Anduril has little to no background in building military soft/hardware [0][1]. The modern day charlatans are alive and living very well.

[0] https://www.techtimes.com/articles/320165/20260711/army-ar-g... [1] https://www.techbuzz.ai/articles/anduril-s-autonomous-weapon...

More comments...