Nothing but respect to TurnTrout for taking an action like this. The world needs more smart people who are willing to stand for what they feel is right, despite the pressures otherwise. Without that occurring more, our species is going to lose many impactful prisoner's dilemmas coming these next two decades.
This raises my respect for AI researchers a little bit too. I have often felt that the entire industry is pretty tainted to the core, and for better or worse that colors my opinion of the researchers.
Maybe I'm in the minority, but I thought it was gross to download pirated art for a student project when I was at Berkeley years ago. So it has been really sad to witness many of the most brilliant minds of this generation answering the siren song of disrespecting the collective effort of others to extract and resell residual value.
I'd guess TurnTrout doesn't agree on that framing, otherwise he probably would not have been at Deep Mind. But clearly he and I agree on other ethical positions; I am nothing but glad to see him stick to his principles here.
Hmm, links for what? My student project? The decisions that face humanity that are pretty clearly modellable as prisoner's dilemmas?
Otherwise not sure what I could cite--I would assume most all on this forum know that AI is trained on the works of other people, without their permission to do so. I guess you could disagree with my framing, but I wouldn't think this requires a citation.
I think maybe my writing wasn't clear, and it sounded like I was referrencing some well known thing that happened at UC Berkeley. I have edited it to read more cleanly!
It wasn't clear to me it was _your_ project nor who pirated it. I was under the impression it was a well known scandal from your original, unedited comment.
Apologies if my comment sounded hostile; I was just asking for a clarification/more information on it.
Props to the author. I left Microsoft due to their work with Israel to spy on Palestinians and record all of their phone conversations (in addition to other IDF collaborations). They ended up walking some of it back, but Satya was complicit.
I salute you for your morality. If most people did this, those companies would change their position very quickly. After all, they only think about money.
Respect for sticking to your beliefs. Just out of curiosity though why do people not want smart AI weapons? I would much rather have an onboard AI that can discriminate between unarmed civilians and military assets, seems irresponsible to not... Is a dumb sea mine that blows up everything somehow better than a smart sea mine that knows to not blow up sometimes?
"You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"
I realize that's not a great argument and was definitely tongue-in-cheek, but given there's still a lot of debate about the accuracy of AI for far more mundane tasks, my personal perspective is that until we have LLMs and such that are truly, demonstrably far more accurate than humans, with true reasoning and judgement capabilities, they don't belong where lives are at stake.
I wouldn't want an LLM-underpinned machine running anesthesia during a surgery; why would I want an LLM-underpinned military apparatus that is deciding the lives of far more? I wouldn't, not in their current state.
In a hypothetical future where we truly trust incredibly smart AIs or LLMs or whatever "smart" technology it is for driving weaponry, okay - if it's truly necessary; I abhor war and the death and destruction wrought by it.
In my mind, though - even if we get to that future where there's some vastly superior technology to the LLMs we have today, which can judge and reason, then I'll have a bunch of other questions, like understanding the motivations of said technology, because I suspect it'll be something much closer to AGI, and that opens a whole separate can of philosophical worms.
Not only that but an AI could follow any orders it is given (whether it hallucinates that's a different story) whereas humans may oppose. The friction of orders having to go through a chain of command through it's completion is a feature not a bug and that adds responsibility. AI can take no responsibility and with moving fast break things smart weapons that responsibility can be shoved under the rug.
> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"
What if it was the target though? AI may be more capable than we're giving it credit for (especially the AI accessible to the US and other governments). The attack on the girls' school coincided with Purim and I don't believe it was a mistake. I think it was the opening salvo by a radically religious Zionists (Christian and Jewish).
I don't think the biggest problem with AI weapons is that they make "mistakes", I think the biggest problem is that they allow people who want to kill civilians the ability to accurately do just that.
So you would prefer weapons that cannot reason at all?
> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"
Most likely this event happened due to a bad targeting system that wasn't smart enough, if it had a better llm underpinning it (assuming it had any) maybe those lives would have been saved. More reason for more smart people like the author to work on these systems.
Mainly I don't want them because I don't want a machine instantly making life or death judgements. It might be fine if it mostly informs a human who ultimately pulls the trigger, but having that failsafe is important in keeping machines from going on killing sprees.
I also want people to be held accountable when they do unjustified killings. AI weapons make it FAR too easy to simply pass off a killing as a "woopsie doodle." It's just not acceptable to say "The algorithm made a mistake, version 23 will do better".
I don't have a problem with the AI providing additional information to it's user, but when that's incorporated into a weapon it's a short distance from that to completely automating the killing.
Responsibility will always have to lie with the user, but if the weapons are more advanced and safer then why is that bad? I like that autonomous vehicles like Waymo are 10x safer then human drivers, even if a 'Machine' is making decisions.
>Just out of curiosity though why do people not want smart AI weapons?
When the decision to kill another human being is made that should be in the hands of a directly accountable other human being, not an unaccountable machine developed in the basement of a private corporation.
And mines, both dumb and smart, in particular anti-personell mines are banned by the Ottawa treaty ratified by 162 countries. It's exactly the autonomous and fundamentally uncontrollable nature of mines, not just that they're dumb, that has produced countless of casualties long after wars were over. Can you tell me that millions of autonomous loitering munitions are not going to end up exactly like those mines still blowing legs off people decades after conflicts are over? And who is responsible then?
They aren’t training smart AI weapons to discriminate between civilians and soldiers. Even so, what happens when the soldiers are disguised as civilians? Or when the enemy forces civilians to serve or be a decoy?
I haven't finished reading, but I should note for the group that the author doesn't represent the Anthropic situation accurately. Emil Michael, the undersecretary of war/defense for research and engineering went on the All-In podcast (1) and explained, for quite some time, exactly what happened. You should go listen to it (2). In essence, in the negotiations, Dario kept coming back saying "well, if you need it, call us, we can redline things as needed". This happened over and over and over. Emil's point was that a major conflict, if it were to occur, might happen on the 30 minute clock of an ICBM, and all due respect to Dario, the national security apparatus just doesn't have an allowance in that 30 minutes to call him for a redline. Emil felt Dario had demonstrated he wanted ultimate control via line item veto, and was willing to trade up to and including the survival of the nation for that veto. And a government cannot be expected to pay for that sort of behavior from an entity domiciled in their jurisdiction.
Now, Dario is going to win a decent slice of the economic pie. But as an military acquisitions matter, I gotta say, I have to agree with the undersecretary's position here, and yeah, it makes sense to document a company's undesirable behavior, and in certain circumstances push that information to others. Not the first time the government has found a company under contract acting in a way that appears to be counterproductive to the government's obligations; there's a whole database full of this stuff (3).
1) Without a doubt, All In is a friendly crowd for Emil, but I think that actually made it easier for him to get more nuanced facts out in this case because he wasn't spending a lot of time defending malignant attacks.
I cannot think of a less trustworthy source of information than Emil Michael on the All-In podcast. Why on Earth would you take his statements at face value? I grant that nobody in this situation is totally trustworthy, but if it's his word against Dario's I'm believing Dario 10 times out of 10, and it's astonishing and dismaying that anyone might do otherwise.
Keep in mind that Emil Michael's job at Uber was to slander and destroy their political enemies. What makes you think he hasn't continued exactly as before?
Even at zero dollars and a billion dollars they feel rare. Honestly I am surprised how many tech people still work for Facebooks and Googles of the world when they actively have used their platforms against most of their values. These are skilled people that can still make a living elsewhere.
And there are others who left years ago and warned us. Back-patting isn't all or nothing, and I think it's reasonable to praise those who left when they saw the writing on the wall more than those who leave only once they see their company materially supporting state killings.
I did not know about TurnTrout till now and thanks to HN community now I know. I am beyond impressed as to how he lives by his principles. I don't think I have this kind of courgae or confidence and thankfully I don't work for Google or have to quit it. On the other hand we have smart people who have choose to work for companies like Palantir and are proud of it.
We know now that Google is a sycophantic company whose DEI initiatives were all fake and dropped once Trump got elected.
This somewhat naive initiative was bound to fail. The good news is that the AI military products won't work, except perhaps for blowing up a girls' school.
Here are CEOs falling over themselves to support Hassabis' regulatory capture proposal:
It's almost like Anduril has little to no background in building military soft/hardware [0][1]. The modern day charlatans are alive and living very well.
If the author were to read these comments, I'd commiserate with them on the disappointment of the utopia left behind in the service of acquiring wealth and power. An abundant solarpunk future enabled and augmented by AI, hyped so well by early Google and Kurzweil, captured as capital to earn a buck for shareholders.
> Plus, a pledge is only worth the credibility behind it. When someone signs “I will not support the development of lethal autonomous weapons,” then stays while their company sells unrestricted AI to a military that wants exactly that, they teach every counterparty a lesson: these safety people will not act, even at their own brightest line. The next commitment they make is worth less. Eventually it’s worth nothing.
This is why I haven’t taken the internally employed (and visibly public) AI safety people seriously for years: they always back down, because they value that fat paycheck and fiscal rewards and associated status over their proclaimed ethical values, red lines, and boundaries. Their actions betray their every word.
That’s also why I push that we remember these names, the ones who fought back as well as those who lied through their teeth. We should remember everyone willing to harm others to protect themselves, and hold them accountable for their misdeeds. In an ideal world, people like Jeff and Sundar and Stuart would be blackballed so hard they’d never be able to get so much as a help desk job ever again, nevermind any position of leadership. It’s why I proclaim them to be monsters, giddily sacrificing the anonymous other for personal vainglory and the avoidance of standing on any sort of principle.
The OP did everything right and demonstrably proved the entire apparatus is rotten to its very core, to the point of infesting and poisoning the very entities professing to stop its worst excesses (IASEAI). Why we give these ghouls the slightest bit of credibility anymore is beyond me.
> Even if Google had adopted your Framework, the Pentagon would have refused
> I agree. xAI would still have given over their AI. But if Google had given signs of independence earlier, it could perhaps have built a coalition with OpenAI and Anthropic.
This is like saying that Google and Apple might build a coalition to prevent App Store regulation. These are competitors, they all see moral flexibility as an advantage. They're not going to take a moralist stance if their federal protection is predicated on federal cooperation. All of these FAANG businesses have already bent the knee in anticipation of this, xAI is just riding the coattails of the federal quid-pro-quo.
When you go to work at a megacorp, you're always leaving your ethics at the door. Yes, there's an attractive pie-in-the-sky fantasy that Apple does care about human rights, or that Google isn't evil, but they're always just lies. I disagree with a lot of GCP's customers, but I'm still shocked that a DeepMind employee would make it this far in the career pipeline before seeing the soylent green get made.
Competitors often wish they didn't have to compete.
For example, a cartel is "a group of independent market participants who collaborate with each other and avoid competition in order to improve profits and dominate the market"
It's something that could happen in this world, through the collective action of employees, that corporate strategy can be changed.
> through the collective action of employees, that corporate strategy can be changed.
I just don't agree with this. I'd like to, but the corporate strategy is not being set by rank-and-file engineers. Sam Altman, Dario Amodei, Elon Musk - these are the executives that are steering the ship. You can change the culture internally and cause a lot of strife, but typically you don't have control over the strategy. If the executive says you're giving API access to the NSA, then that's what is going to happen.
... I'm sure I'll get flamed into oblivion for this but it's weird to me how the zeitgeist is anti-colonialism but also against enforcing borders and national sovereignty. I guess maybe they are okay with it unless you're a western nation. Whatever, there's no room for nuanced opinions anymore in modern online discourse.
I'll admit to being a weird outlier on this. I may not like what certain parts of the government are doing but I'd go work for the Voldemort companies in a heartbeat -- they just require in-office and aren't anywhere near me. I'd rather my nation develop the best technologies than let other nations do it.
Just look at how far behind the eurozone is by not making the right investments.
> I'm sure I'll get flamed into oblivion for this but it's weird to me how the zeitgeist is anti-colonialism but also against enforcing borders and national sovereignty
Let me try a reply that is perhaps a little more constructive than where this discussion went.
I think most folks would frame home rule and freedom of movement in the context of basic individual rights, not national rights. I suggest you might fare better explaining some context about why you believe they should care about nation states to begin with
"Freedom of movement" is a fairly loaded term, but is defined in the Universal Declaration of Human Rights is:
- "Everyone has the right to freedom of movement and residence within the borders of each state."
- "Everyone has the right to leave any country, including his own, and to return to his country."
Freedom of movement isn't the right to go live anywhere you want across any national jurisdiction (although some people/organizations would disagree, but that just doesn't survive any legal test). Every nation on the planet has very explicit rules around this. Even the unrecognized ones.
I guess I don't necessarily agree with elected officials systematically not enforcing laws that they don't agree with. It's hard to be rigid about this when there's things like Executive Orders, but I'm very pro "Change the law". And admittedly that's very difficult and it should be so -- new law should be hard to get to.
This process happens all the way from the local to national level, but we have terrible governance. Every structure seems to be abdicating its responsibility and overreaching in one way or another. I think it's precisely in these conditions that I think leaning into rule of law is more important than ever.
Otherwise we're just playing Whose Country Is It Anyway? Where the rules are made up and the laws don't matter.
I mean this sincerely, may you never find employment again. I hope you wash out of the industry due to whatever festering mental issues finally take hold and render you unable to carry out evil for a better paycheck. Goddamn you and your ilk.
Same to you I guess? The world is not a black and white place. Rigorous self-defense is not evil.
I chose to be vulnerable and honest knowing what kind of responses I would get and the guidelines of this board call for assuming good faith. It's discourse like yours that is inflaming and ruining our society.
I didn't even endorse any specific actions here, I'm just not meeting your moral threshold based on your interpretation of some pretty thinly-detailed comments that I made.
You also can't influence the game if you're not a player so opting out of working in the entire defense industry is probably against your interests.
> some comment you edited out where you said I'm heartless and spineless.
And I know what it's like to live in a place without adequate law enforcement and/or military.
Personal insults and calling me mentally ill. Are you projecting?
Apparently willingness to work at Palantir or Anduril or other defense contractors makes me irredeemably evil. I'm sure you're real fun at parties.
Tbh, I'm glad though. I'm really great at what I do and have had a long and successful career. We need more hiring filters and I don't want coworkers with deeply skewed perspectives, inadequate objectivity and who cannot coexist with those of different world-views. Self-filtering is the cheapest and best kind.
> I wanted AI ethics commitments to hold under pressure.
LLM chatbots, they way they are trained and have data collected for them today, are fundamentally unethical, regardless of whether Google sells services to the DHS.
No less importantly, Google's is a fundamentally unethical entity. After all, it says so right in its motto: "Do Evil".
Ok, that's not it's motto, it's just a pun on the old motto Google dropped when it became too starkly opposed to its practices, of mass surveillance and manipulation and censorship of search and content discovery results for commercial and political purposes, on its behalf and that of governments. This has been widely reported upon.
I will just give a few links regarding Google's complicity in the ongoing genocide in Gaza, as described by the UN special rapporteur for the Palestinian Occupied Territories:
I mean - the idea that any company employing non-trivial amounts of people "cares" in any sane sense of the word "care" is absurd.
Who is supposed to "care" about you? A single person, definitely not the CEO, is going to individually care about 20k+ workers, especially not 200k+ workers.
Even if you assumed an altruistic HR department, it's still going to be a faceless blob when you're a 100k+ company.
> Jeff Dean ... and other senior employees pledged to “neither participate in nor support the development, manufacture, trade, or use of lethal autonomous weapons.” Google signed the classified deal, yet they remain.
> I think he could have stopped the deal, yet he did not. He remains, yet I think he should not.
Is Jeff Dean still considered a saint at Google? If so, how come this doesn't change that? The amicus was enough?
I mean, on the one hand, I support this. But this is going pretty far. When it comes to non-US police and military, if your threshold is killing 2 innocents (and the officers getting away with it), then you can't do business almost anywhere ...
With essentially any country, and even vaguely similar circumstances
And this is just a random example country (with excellent beer), you will find worse than this in most/all European countries, especially in the last 10 years or so.
This is pretty different - in both of those, the officer faced consequences. Unidentified masked ICE officers have killed about a person a month with no consequences, many times over a civil concern (immigration status) or just getting annoyed with protesters who broke no laws. ICE is effectively a lethal extrajudicial force with no means of seeking accountability when they kill.
On top of that, they are acting aggressively and violently in broad daylight solely to terrorize immigrant communities. The chilling effect is very visible in hospital systems right now - I have seen far less Hispanics in the hospital for medical emergencies, and that includes people who are in this country legally.
your straw man is misrepresenting the scale of violence ICE has inflicted on communities across the country. the article states it was after Alex Pirettis death that he was inspired to act. ICE has killed many people across the country during activities which they have no business conducting. A man in Houston was killed last week, then days later, another in Maine. They’ve shot citizens and fabricated evidence to cover up their actions in multiple instances.
Wake the fuck up and stop trying to whatabout your way to a more comfortable mental state.
at a micro/personal level, absolutely keep having the tough conversation that it’s not okay if the topics arise in your circles. encourage others to reflect on the issue more if they seem apathetic, and paint various pictures to help folks overcome their own biases.
conjuring empathy in others is not easy, but it is worth it.
This is a long post about someone who has very obviously just gotten into politics. It is good for people to try and see how to impart change. Here are some constructive critical questions for the author:
1. Why no mention of No Tech For Apartheid or Google Workers United, who have been doing similar work for years?
2. What about all of the other police, DHS, and military contracts Google has been a part of? Did this problem really just start with the second (not even the first!) Trump presidency?
3. What does a focus on exclusively those at the top levels of a hierarchy, with minimal focus on incentive structures and wider systems, say about your theory of change? Was there a power analysis done, or was it assumed that "big title" = "powerful"?
Side Note: Incredibly insulting of James Dean to say email 3 CEOs.
I don't hold it against people for only caring about one or two issues. In addition to many plausible explanations, people like OP who take a stand are already doing so much.
I for sure agree about encouraging people to dig deeper and wider, and that absolutely none of this is new or is occurring in a vacuum, but (and I don't say this to single you out, this is just a long-held feeling I've had) I do also think it's important to encourage and commend anyone who reaches these conclusions, however late or nascent they might be. Especially someone with the backbone to speak out at and then quit a cushy, prestigious job.
Something something every journey, something something single step. For the author (and for all of us, really) I hope it's one of many. And I think they should be proud of this particular step :)
Regardless of whether AI is involved, many people simply don't want to work on weapons systems, or on non-weapons technologies that are intended to become part of a weapons system.
I think there's some nuance here that a lot of these convos dont make plain - are we talking about just using 'ai' in the military broadly, or are we specifically talking about fully autonomous weapons systems?
In the former case, I would agree with you completely, I havent heard any arguments beyond 'I dont like working on military stuff'. But if we're talking fully autonomous weapons, that's a different story. And further muddying the waters is the fact that the former is obviously a step along the path to the latter.
AI helps people scale. If being "on target" means that you find people when they're eating dinner so you can kill their entire family, then using AI to scale that up is a terrible thing. That's exactly what Israel did with Lavender. They literally had a kill list called "Where's daddy?".
AI is the end of the funnel and I actually think the mass surveillance at the beginning of the funnel is a bigger issue. The concentration of power is the problem. If you or I could defend ourselves against state level actors with AI weapons, then they would be a good thing (imho). AI and mass surveillance at the state level create a power imbalance that is a major threat to human rights and should be resisted as much as possible.
It does appear to be anti-democratic given everyone knew Trump’s platform when he was voted in. However the great unifying fact of politics is that everyone only really believes in democracy when they win.
Nothing but respect to TurnTrout for taking an action like this. The world needs more smart people who are willing to stand for what they feel is right, despite the pressures otherwise. Without that occurring more, our species is going to lose many impactful prisoner's dilemmas coming these next two decades.
This raises my respect for AI researchers a little bit too. I have often felt that the entire industry is pretty tainted to the core, and for better or worse that colors my opinion of the researchers.
Maybe I'm in the minority, but I thought it was gross to download pirated art for a student project when I was at Berkeley years ago. So it has been really sad to witness many of the most brilliant minds of this generation answering the siren song of disrespecting the collective effort of others to extract and resell residual value.
I'd guess TurnTrout doesn't agree on that framing, otherwise he probably would not have been at Deep Mind. But clearly he and I agree on other ethical positions; I am nothing but glad to see him stick to his principles here.
Apologies, but it would be good to add links for your anecdotes.
Not all of the readers of your comment have the appropriate context and know what you're talking about. I certainly don't.
Hmm, links for what? My student project? The decisions that face humanity that are pretty clearly modellable as prisoner's dilemmas?
Otherwise not sure what I could cite--I would assume most all on this forum know that AI is trained on the works of other people, without their permission to do so. I guess you could disagree with my framing, but I wouldn't think this requires a citation.
I think maybe my writing wasn't clear, and it sounded like I was referrencing some well known thing that happened at UC Berkeley. I have edited it to read more cleanly!
Yes; the student project you were talking about.
It wasn't clear to me it was _your_ project nor who pirated it. I was under the impression it was a well known scandal from your original, unedited comment.
Apologies if my comment sounded hostile; I was just asking for a clarification/more information on it.
No worries! Thanks for engaging. Fun to have the rare kindly-resolved internet discussion.
Props to the author. I left Microsoft due to their work with Israel to spy on Palestinians and record all of their phone conversations (in addition to other IDF collaborations). They ended up walking some of it back, but Satya was complicit.
https://www.theguardian.com/world/2025/sep/25/microsoft-bloc...
I salute you for your morality. If most people did this, those companies would change their position very quickly. After all, they only think about money.
Respect!
Respect for sticking to your beliefs. Just out of curiosity though why do people not want smart AI weapons? I would much rather have an onboard AI that can discriminate between unarmed civilians and military assets, seems irresponsible to not... Is a dumb sea mine that blows up everything somehow better than a smart sea mine that knows to not blow up sometimes?
"You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"
I realize that's not a great argument and was definitely tongue-in-cheek, but given there's still a lot of debate about the accuracy of AI for far more mundane tasks, my personal perspective is that until we have LLMs and such that are truly, demonstrably far more accurate than humans, with true reasoning and judgement capabilities, they don't belong where lives are at stake.
I wouldn't want an LLM-underpinned machine running anesthesia during a surgery; why would I want an LLM-underpinned military apparatus that is deciding the lives of far more? I wouldn't, not in their current state.
In a hypothetical future where we truly trust incredibly smart AIs or LLMs or whatever "smart" technology it is for driving weaponry, okay - if it's truly necessary; I abhor war and the death and destruction wrought by it.
In my mind, though - even if we get to that future where there's some vastly superior technology to the LLMs we have today, which can judge and reason, then I'll have a bunch of other questions, like understanding the motivations of said technology, because I suspect it'll be something much closer to AGI, and that opens a whole separate can of philosophical worms.
Not only that but an AI could follow any orders it is given (whether it hallucinates that's a different story) whereas humans may oppose. The friction of orders having to go through a chain of command through it's completion is a feature not a bug and that adds responsibility. AI can take no responsibility and with moving fast break things smart weapons that responsibility can be shoved under the rug.
> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"
What if it was the target though? AI may be more capable than we're giving it credit for (especially the AI accessible to the US and other governments). The attack on the girls' school coincided with Purim and I don't believe it was a mistake. I think it was the opening salvo by a radically religious Zionists (Christian and Jewish).
I don't think the biggest problem with AI weapons is that they make "mistakes", I think the biggest problem is that they allow people who want to kill civilians the ability to accurately do just that.
So you would prefer weapons that cannot reason at all?
> "You're absolutely right, that wasn't a military target—it was actually a girls school. It won't happen again!"
Most likely this event happened due to a bad targeting system that wasn't smart enough, if it had a better llm underpinning it (assuming it had any) maybe those lives would have been saved. More reason for more smart people like the author to work on these systems.
Mainly I don't want them because I don't want a machine instantly making life or death judgements. It might be fine if it mostly informs a human who ultimately pulls the trigger, but having that failsafe is important in keeping machines from going on killing sprees.
I also want people to be held accountable when they do unjustified killings. AI weapons make it FAR too easy to simply pass off a killing as a "woopsie doodle." It's just not acceptable to say "The algorithm made a mistake, version 23 will do better".
I don't have a problem with the AI providing additional information to it's user, but when that's incorporated into a weapon it's a short distance from that to completely automating the killing.
That's why I'm completely against AI weapons.
Responsibility will always have to lie with the user, but if the weapons are more advanced and safer then why is that bad? I like that autonomous vehicles like Waymo are 10x safer then human drivers, even if a 'Machine' is making decisions.
>Just out of curiosity though why do people not want smart AI weapons?
When the decision to kill another human being is made that should be in the hands of a directly accountable other human being, not an unaccountable machine developed in the basement of a private corporation.
And mines, both dumb and smart, in particular anti-personell mines are banned by the Ottawa treaty ratified by 162 countries. It's exactly the autonomous and fundamentally uncontrollable nature of mines, not just that they're dumb, that has produced countless of casualties long after wars were over. Can you tell me that millions of autonomous loitering munitions are not going to end up exactly like those mines still blowing legs off people decades after conflicts are over? And who is responsible then?
The mines got banned because they were dumb...
They aren’t training smart AI weapons to discriminate between civilians and soldiers. Even so, what happens when the soldiers are disguised as civilians? Or when the enemy forces civilians to serve or be a decoy?
Because it's going to be used in any possible way, and certainly not for the good of the people, as imply in your "naive" false dichotomy.
I haven't finished reading, but I should note for the group that the author doesn't represent the Anthropic situation accurately. Emil Michael, the undersecretary of war/defense for research and engineering went on the All-In podcast (1) and explained, for quite some time, exactly what happened. You should go listen to it (2). In essence, in the negotiations, Dario kept coming back saying "well, if you need it, call us, we can redline things as needed". This happened over and over and over. Emil's point was that a major conflict, if it were to occur, might happen on the 30 minute clock of an ICBM, and all due respect to Dario, the national security apparatus just doesn't have an allowance in that 30 minutes to call him for a redline. Emil felt Dario had demonstrated he wanted ultimate control via line item veto, and was willing to trade up to and including the survival of the nation for that veto. And a government cannot be expected to pay for that sort of behavior from an entity domiciled in their jurisdiction.
Now, Dario is going to win a decent slice of the economic pie. But as an military acquisitions matter, I gotta say, I have to agree with the undersecretary's position here, and yeah, it makes sense to document a company's undesirable behavior, and in certain circumstances push that information to others. Not the first time the government has found a company under contract acting in a way that appears to be counterproductive to the government's obligations; there's a whole database full of this stuff (3).
1) Without a doubt, All In is a friendly crowd for Emil, but I think that actually made it easier for him to get more nuanced facts out in this case because he wasn't spending a lot of time defending malignant attacks.
2) This link should jump to the relevant segment https://www.youtube.com/watch?v=gzwRflcLPAA&t=2479
3) http://www.ppirs.gov/
That doesn't address the supply chain risk designation? Or the ban on all work with claude for contractors, incl. non-government work?
Even if they disagreed, this was clearly retaliatory for not capitulating. The government could have just decided to go with a different provider.
I cannot think of a less trustworthy source of information than Emil Michael on the All-In podcast. Why on Earth would you take his statements at face value? I grant that nobody in this situation is totally trustworthy, but if it's his word against Dario's I'm believing Dario 10 times out of 10, and it's astonishing and dismaying that anyone might do otherwise.
Keep in mind that Emil Michael's job at Uber was to slander and destroy their political enemies. What makes you think he hasn't continued exactly as before?
Wait. Do you trust what he said?
Principled people have become so rare
Only when the bank account reaches 7 - 8 figures.
Even at zero dollars and a billion dollars they feel rare. Honestly I am surprised how many tech people still work for Facebooks and Googles of the world when they actively have used their platforms against most of their values. These are skilled people that can still make a living elsewhere.
No need to downplay it. There are many for whom it's never enough.
And there are others who left years ago and warned us. Back-patting isn't all or nothing, and I think it's reasonable to praise those who left when they saw the writing on the wall more than those who leave only once they see their company materially supporting state killings.
Fair.
I did not know about TurnTrout till now and thanks to HN community now I know. I am beyond impressed as to how he lives by his principles. I don't think I have this kind of courgae or confidence and thankfully I don't work for Google or have to quit it. On the other hand we have smart people who have choose to work for companies like Palantir and are proud of it.
This is inspiring, thank you to the author, it must have a hard piece to write.
We know now that Google is a sycophantic company whose DEI initiatives were all fake and dropped once Trump got elected.
This somewhat naive initiative was bound to fail. The good news is that the AI military products won't work, except perhaps for blowing up a girls' school.
Here are CEOs falling over themselves to support Hassabis' regulatory capture proposal:
https://xcancel.com/sundarpichai/status/2077086951833063580#...
https://xcancel.com/satyanadella/status/2077063479232795024#...
https://xcancel.com/elonmusk/status/2077415601610297535#m
It is an exclusive club and we are not part of it.
Saying AI military products wont work is the dumbest thing i've heard in awhile
Search for Anduril drone failures in Ukraine and several Lattice software failures. And perhaps capitalize "I".
How is Maven working in Iran?
Thats like saying cars wont work because they get flat tires
It's almost like Anduril has little to no background in building military soft/hardware [0][1]. The modern day charlatans are alive and living very well.
[0] https://www.techtimes.com/articles/320165/20260711/army-ar-g... [1] https://www.techbuzz.ai/articles/anduril-s-autonomous-weapon...
If the author were to read these comments, I'd commiserate with them on the disappointment of the utopia left behind in the service of acquiring wealth and power. An abundant solarpunk future enabled and augmented by AI, hyped so well by early Google and Kurzweil, captured as capital to earn a buck for shareholders.
Mad props for pushing as far as they could push.
I think a man who makes that kind of choice is seriously admirable. I could never have made that choice myself.
> Plus, a pledge is only worth the credibility behind it. When someone signs “I will not support the development of lethal autonomous weapons,” then stays while their company sells unrestricted AI to a military that wants exactly that, they teach every counterparty a lesson: these safety people will not act, even at their own brightest line. The next commitment they make is worth less. Eventually it’s worth nothing.
This is why I haven’t taken the internally employed (and visibly public) AI safety people seriously for years: they always back down, because they value that fat paycheck and fiscal rewards and associated status over their proclaimed ethical values, red lines, and boundaries. Their actions betray their every word.
That’s also why I push that we remember these names, the ones who fought back as well as those who lied through their teeth. We should remember everyone willing to harm others to protect themselves, and hold them accountable for their misdeeds. In an ideal world, people like Jeff and Sundar and Stuart would be blackballed so hard they’d never be able to get so much as a help desk job ever again, nevermind any position of leadership. It’s why I proclaim them to be monsters, giddily sacrificing the anonymous other for personal vainglory and the avoidance of standing on any sort of principle.
The OP did everything right and demonstrably proved the entire apparatus is rotten to its very core, to the point of infesting and poisoning the very entities professing to stop its worst excesses (IASEAI). Why we give these ghouls the slightest bit of credibility anymore is beyond me.
> Even if Google had adopted your Framework, the Pentagon would have refused
> I agree. xAI would still have given over their AI. But if Google had given signs of independence earlier, it could perhaps have built a coalition with OpenAI and Anthropic.
This is like saying that Google and Apple might build a coalition to prevent App Store regulation. These are competitors, they all see moral flexibility as an advantage. They're not going to take a moralist stance if their federal protection is predicated on federal cooperation. All of these FAANG businesses have already bent the knee in anticipation of this, xAI is just riding the coattails of the federal quid-pro-quo.
When you go to work at a megacorp, you're always leaving your ethics at the door. Yes, there's an attractive pie-in-the-sky fantasy that Apple does care about human rights, or that Google isn't evil, but they're always just lies. I disagree with a lot of GCP's customers, but I'm still shocked that a DeepMind employee would make it this far in the career pipeline before seeing the soylent green get made.
Competitors often wish they didn't have to compete.
For example, a cartel is "a group of independent market participants who collaborate with each other and avoid competition in order to improve profits and dominate the market"
It's something that could happen in this world, through the collective action of employees, that corporate strategy can be changed.
I hope he becomes the seed crystal around which crystallization of peaceful use of AI happens.
> through the collective action of employees, that corporate strategy can be changed.
I just don't agree with this. I'd like to, but the corporate strategy is not being set by rank-and-file engineers. Sam Altman, Dario Amodei, Elon Musk - these are the executives that are steering the ship. You can change the culture internally and cause a lot of strife, but typically you don't have control over the strategy. If the executive says you're giving API access to the NSA, then that's what is going to happen.
apple and google are regularly on the same side against eu regulations
On the upside, I guess maybe DeepMind is hiring.
... I'm sure I'll get flamed into oblivion for this but it's weird to me how the zeitgeist is anti-colonialism but also against enforcing borders and national sovereignty. I guess maybe they are okay with it unless you're a western nation. Whatever, there's no room for nuanced opinions anymore in modern online discourse.
I'll admit to being a weird outlier on this. I may not like what certain parts of the government are doing but I'd go work for the Voldemort companies in a heartbeat -- they just require in-office and aren't anywhere near me. I'd rather my nation develop the best technologies than let other nations do it.
Just look at how far behind the eurozone is by not making the right investments.
> I'm sure I'll get flamed into oblivion for this but it's weird to me how the zeitgeist is anti-colonialism but also against enforcing borders and national sovereignty
Let me try a reply that is perhaps a little more constructive than where this discussion went.
I think most folks would frame home rule and freedom of movement in the context of basic individual rights, not national rights. I suggest you might fare better explaining some context about why you believe they should care about nation states to begin with
Fair comment and thank you.
"Freedom of movement" is a fairly loaded term, but is defined in the Universal Declaration of Human Rights is: - "Everyone has the right to freedom of movement and residence within the borders of each state." - "Everyone has the right to leave any country, including his own, and to return to his country."
Freedom of movement isn't the right to go live anywhere you want across any national jurisdiction (although some people/organizations would disagree, but that just doesn't survive any legal test). Every nation on the planet has very explicit rules around this. Even the unrecognized ones.
I guess I don't necessarily agree with elected officials systematically not enforcing laws that they don't agree with. It's hard to be rigid about this when there's things like Executive Orders, but I'm very pro "Change the law". And admittedly that's very difficult and it should be so -- new law should be hard to get to.
This process happens all the way from the local to national level, but we have terrible governance. Every structure seems to be abdicating its responsibility and overreaching in one way or another. I think it's precisely in these conditions that I think leaning into rule of law is more important than ever.
Otherwise we're just playing Whose Country Is It Anyway? Where the rules are made up and the laws don't matter.
I mean this sincerely, may you never find employment again. I hope you wash out of the industry due to whatever festering mental issues finally take hold and render you unable to carry out evil for a better paycheck. Goddamn you and your ilk.
Same to you I guess? The world is not a black and white place. Rigorous self-defense is not evil.
I chose to be vulnerable and honest knowing what kind of responses I would get and the guidelines of this board call for assuming good faith. It's discourse like yours that is inflaming and ruining our society.
I didn't even endorse any specific actions here, I'm just not meeting your moral threshold based on your interpretation of some pretty thinly-detailed comments that I made.
You also can't influence the game if you're not a player so opting out of working in the entire defense industry is probably against your interests.
make no mistake, it’s mindsets like yours that put us on the path we currently tread.
> I'd go work for the Voldemort companies in a heartbeat […]
being honest and vulnerable about being a shitty person does not beget or earn you any kind of mutual respect. we are simply not the same.
> some comment you edited out where you said I'm heartless and spineless.
And I know what it's like to live in a place without adequate law enforcement and/or military.
Personal insults and calling me mentally ill. Are you projecting?
Apparently willingness to work at Palantir or Anduril or other defense contractors makes me irredeemably evil. I'm sure you're real fun at parties.
Tbh, I'm glad though. I'm really great at what I do and have had a long and successful career. We need more hiring filters and I don't want coworkers with deeply skewed perspectives, inadequate objectivity and who cannot coexist with those of different world-views. Self-filtering is the cheapest and best kind.
> Apparently willingness to work at Palantir or Anduril or other defense contractors makes me irredeemably evil.
hey, look! it’s learning!
oh my god, your edits are so good.
> I don't want coworkers with deeply skewed perspectives, inadequate objectivity and who cannot coexist with those of different world-views
and I don’t want coworkers that are okay with killing children in Gaza.
am I the only one that finds it very hard to read this font?
> I wanted AI ethics commitments to hold under pressure.
LLM chatbots, they way they are trained and have data collected for them today, are fundamentally unethical, regardless of whether Google sells services to the DHS.
No less importantly, Google's is a fundamentally unethical entity. After all, it says so right in its motto: "Do Evil".
Ok, that's not it's motto, it's just a pun on the old motto Google dropped when it became too starkly opposed to its practices, of mass surveillance and manipulation and censorship of search and content discovery results for commercial and political purposes, on its behalf and that of governments. This has been widely reported upon.
I will just give a few links regarding Google's complicity in the ongoing genocide in Gaza, as described by the UN special rapporteur for the Palestinian Occupied Territories:
https://novaramedia.com/2025/07/02/tech-giants-and-british-b...
(or get the report itself: https://www.ohchr.org/sites/default/files/documents/hrbodies...)
If you do not like what Google is doing, just leave as I said before [0]. They don't care about you.
[0] https://news.ycombinator.com/item?id=47931336
I mean - the idea that any company employing non-trivial amounts of people "cares" in any sane sense of the word "care" is absurd.
Who is supposed to "care" about you? A single person, definitely not the CEO, is going to individually care about 20k+ workers, especially not 200k+ workers.
Even if you assumed an altruistic HR department, it's still going to be a faceless blob when you're a 100k+ company.
> Jeff Dean ... and other senior employees pledged to “neither participate in nor support the development, manufacture, trade, or use of lethal autonomous weapons.” Google signed the classified deal, yet they remain.
> I think he could have stopped the deal, yet he did not. He remains, yet I think he should not.
Is Jeff Dean still considered a saint at Google? If so, how come this doesn't change that? The amicus was enough?
tldr: scientist discovers reality
If half of us were half as principled as this guy, the world would be a much better place.
No, they would just walked all over by the 2% who have no principles whatsoever. Kind of like what's happening already.
Principles are only good up until they enable you to be systematically victimized.
Don't play fair in a game others cheat.
tldr: scientist tell us that defending one's values needs courage and strong will.
I mean, on the one hand, I support this. But this is going pretty far. When it comes to non-US police and military, if your threshold is killing 2 innocents (and the officers getting away with it), then you can't do business almost anywhere ...
With essentially any country, and even vaguely similar circumstances
https://www.theguardian.com/world/2021/nov/05/belgian-police...
https://www.theguardian.com/world/2020/aug/20/belgian-police...
And this is just a random example country (with excellent beer), you will find worse than this in most/all European countries, especially in the last 10 years or so.
This is pretty different - in both of those, the officer faced consequences. Unidentified masked ICE officers have killed about a person a month with no consequences, many times over a civil concern (immigration status) or just getting annoyed with protesters who broke no laws. ICE is effectively a lethal extrajudicial force with no means of seeking accountability when they kill.
On top of that, they are acting aggressively and violently in broad daylight solely to terrorize immigrant communities. The chilling effect is very visible in hospital systems right now - I have seen far less Hispanics in the hospital for medical emergencies, and that includes people who are in this country legally.
IOW: "if I have to kill only 2-3 people to make the business I want to make, then, bah, so be it".
its more like: good luck finding clients who don't kill anyone somewhere deep inside their supply chain.
What are dishonest mischaracterization of the issue at hand, shame on you
your straw man is misrepresenting the scale of violence ICE has inflicted on communities across the country. the article states it was after Alex Pirettis death that he was inspired to act. ICE has killed many people across the country during activities which they have no business conducting. A man in Houston was killed last week, then days later, another in Maine. They’ve shot citizens and fabricated evidence to cover up their actions in multiple instances.
Wake the fuck up and stop trying to whatabout your way to a more comfortable mental state.
> They’ve shot citizens and fabricated evidence to cover up their actions in multiple instances.
typical police activity protected by impunity in this country.
i don’t think statements like this help reinforce how serious the situation is; rather, it subtly normalizes it.
Normal/not normal is a matter of opinion.
It sounds like it is normal for majority of political elite as well as voters, since this is not top of political agenda in this country.
It is not normal from my point of view, but I am not sure what can I do about this.
at a micro/personal level, absolutely keep having the tough conversation that it’s not okay if the topics arise in your circles. encourage others to reflect on the issue more if they seem apathetic, and paint various pictures to help folks overcome their own biases.
conjuring empathy in others is not easy, but it is worth it.
This is a long post about someone who has very obviously just gotten into politics. It is good for people to try and see how to impart change. Here are some constructive critical questions for the author:
1. Why no mention of No Tech For Apartheid or Google Workers United, who have been doing similar work for years?
2. What about all of the other police, DHS, and military contracts Google has been a part of? Did this problem really just start with the second (not even the first!) Trump presidency?
3. What does a focus on exclusively those at the top levels of a hierarchy, with minimal focus on incentive structures and wider systems, say about your theory of change? Was there a power analysis done, or was it assumed that "big title" = "powerful"?
Side Note: Incredibly insulting of James Dean to say email 3 CEOs.
I don't hold it against people for only caring about one or two issues. In addition to many plausible explanations, people like OP who take a stand are already doing so much.
I for sure agree about encouraging people to dig deeper and wider, and that absolutely none of this is new or is occurring in a vacuum, but (and I don't say this to single you out, this is just a long-held feeling I've had) I do also think it's important to encourage and commend anyone who reaches these conclusions, however late or nascent they might be. Especially someone with the backbone to speak out at and then quit a cushy, prestigious job.
Something something every journey, something something single step. For the author (and for all of us, really) I hope it's one of many. And I think they should be proud of this particular step :)
It is honorable but ultimately hasn’t democracy spoken wrt the issues mentioned in the post?
For better or worse millions of Americans voted for the guy doing the deportations.
I also find it difficult to reconcile not using AI for weapons. If war is inevitable AI presumably would at least ensure you are on target.
Agreed, I feel like the people who say no to AI weapons haven't actually presented a real argument (that I have heard) besides terminator bad
Regardless of whether AI is involved, many people simply don't want to work on weapons systems, or on non-weapons technologies that are intended to become part of a weapons system.
Thats fine, but a serious luxury belief IMO
I think there's some nuance here that a lot of these convos dont make plain - are we talking about just using 'ai' in the military broadly, or are we specifically talking about fully autonomous weapons systems?
In the former case, I would agree with you completely, I havent heard any arguments beyond 'I dont like working on military stuff'. But if we're talking fully autonomous weapons, that's a different story. And further muddying the waters is the fact that the former is obviously a step along the path to the latter.
AI helps people scale. If being "on target" means that you find people when they're eating dinner so you can kill their entire family, then using AI to scale that up is a terrible thing. That's exactly what Israel did with Lavender. They literally had a kill list called "Where's daddy?".
https://news.ycombinator.com/item?id=39918245
As opposed to what though? Again, I assert violence is inevitable.
AI is the end of the funnel and I actually think the mass surveillance at the beginning of the funnel is a bigger issue. The concentration of power is the problem. If you or I could defend ourselves against state level actors with AI weapons, then they would be a good thing (imho). AI and mass surveillance at the state level create a power imbalance that is a major threat to human rights and should be resisted as much as possible.
It does appear to be anti-democratic given everyone knew Trump’s platform when he was voted in. However the great unifying fact of politics is that everyone only really believes in democracy when they win.
>AI presumably would at least ensure you are on target.
That is a big presumption.