Which regulators though? There is no AI regulator that I know of, and I'm not sure they should be one. Exaggerated marketing blurb is pretty legal, right?
What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.
As you said though, that wouldn’t have the desired effect.
These tools literally can only do what you give them access to, give them access to general tools like a linux shell and they can access to everything that comes with that obviously. The security industry figured out sandboxing and isolation a long time ago. You wanna be 100% sure it doesn't break out on open internet? Run it on a airgapped machine and don't give it network connections. OpenAI is (sadly) demonstrating they aren't responsible nor knowledgeable enough to actually run these experiments.
Asking a agent harness to try whatever it wants to achieve some results, without guardrails, while running in lightweight isolation on 3rd party infrastructure? Feels like they didn't even try, people should be held responsible for this.
It doesn’t behave like a super intelligence because it is not.
One of the key strengths agents have is they just keep going and going, and for cyberattacks that is often unreasonably effective. It is like a barely more aware fuzzing.
> This will not happen though, because these stories are marketing.
The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing.
It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them".
OpenAI could report that its AI started spontaneously generating illegal porn and sending it to people and you'd still think it was a marketing stunt.
I wouldn't describe this stunt as marketing: I would describe it as a bungled attempt to manufacture evidence to get regulatory capture, which OpenAI desperately needs. (Bungled, because Hugging Face fended off the intrusion and got their story out faster than OpenAI did.)
It is because a loud part of the doomerist contingent believe only a major disaster will provide the motivation for regulation, so they are practically hoping to cultivate a major disaster. (Given their preoccupation with bioweapons that probably means a pandemic).
This is based on similar thinking to how the world would not have considered nuclear weapons a major threat if they had forever stayed unused.
The irony of the doomer position is it achieves exactly their supposed nightmare scenario of disaster leading to authoritarian world government without any of the supposed benefits, the twist being they get to be the authoritarian world government so they are ok with it.
So your assessment of the "doomerist" position is that they believe people aren't taking AI risks seriously enough, and that only a large enough catastrophe will wake people up, and your position is we should... Ignore increasingly blatant minor catastrophes to spite them?
This isn’t a minor catastrophe. It is OpenAI being utterly irresponsible by not sandboxing their testing appropriately, to the point it has to be deliberate to provoke uneducated hysteria, which it obviously, sadly, achieves.
Sandboxing processes on networks is not exactly rocket science, and it is something their existing products would readily help them setup.
I have no idea where you’re getting your information from. I’m extremely skeptical of AI and see it as an anti-human technology we would be better without. I believe we are in a massive economical AI bubble that will eventually collapse, unless a financial miracle happens and it is somehow deflated extremely carefully. So I assume you would consider me a doomer. What do biological weapons and world government have to do with that? I’m sorry but your comment reads fairly unhinged to me
> They should shut them down immediately then investigate.
Yeah, I don't understand either. If there was a "hitman for hire" service on the clearweb, the police would shut it down first, then ask questions. Now we have a huge company effectively letting AI agents without guardrails run amok on 3rd party infrastructure, and the police is doing nothing?
They somehow forgot to mention that Hugging Face tried to use frontier models to analyze the attack but all models rejected. They had to use GLM 5.2 deployed locally.
Meanwhile Anthropic is scaremongering about China and also calling for more AI regulation (specifically mandatory safety testing of all models including open model):
I think it has plenty to do with that. The big frontier labs have been crying every step of the way, yelling and screaming to the world about how dangerous LLMs are and how quickly it all can end if they get out of control. None of that's happened; all that's happened so far is regular capitalism stuff. If LLMs ever do actually get out of control to that point, that would be the wolf, but we've all been ignoring the crying for so long that, well, you know.
They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. They've been saying "but we actually mean it this time" every single time. They've exhausted pretty much every possible route for it. The wolf is not real. It hasn't been real. For all we know it's on the horizon, but nobody is going to listen to them in order to know that. And when they say "I told you so", well they've been saying that too over and over about small things, so nobody's still going to bat an eye.
At this point, no one will believe it until they see it with their own eyes.
> They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real.
1.) There was no change in how real the wolf is.
2.) They, literally they, are the wolf. Not "roque ai" or some other bullshit.
That's exactly my point. It hasn't changed for so long, despite all their crying and screaming, that I don't believe them either.
To be perfectly clear, "the wolf" here would be AI becoming a genuine existential threat to humanity that cannot be contained or controlled in a meaningful sense. That's what I refer to in my original comment, and also what the labs have been crying so much about.
A frontier lab today is not that threat because they can choose to shut off their systems at any time. It currently remains a people problem and not a technology problem. There's no self-improving technology that can also replicate itself to evade containment, yet.
But they've been crying about the possibility, constantly, incessantly, and of course saying they need more money about it too. That's just what corporations do. But because they've been crying so much about something that literally does not exist, their cries have just become pointless noise to me. I have considered them eyeroll-worthy marketing stunts for a while.
If one day "the wolf" actually happens, I could not learn about it from the frontier labs themselves, because I would not believe them. They've cried about nothing for so long that I've stopped listening. That's what I mean about having to see it with my own eyes. Until that point, their crying is just pointless, meaningless noise, and there's nothing they can do to change that now.
No longer well anybody be wasting their lives doing data entry. That absolutely hellish job is practically dead now. And there are thousands of low hanging fruit type cases AI fixes. Ai is the new digital dogsbody. You might not think that's useful but the rest of the world does.
Just like cars, AI will kill some of us, maybe thousands every year. But you won't see it disappear with that much genuine benefit currently netting off the harm.
I bet in the future, for both cars and llms, a new and safer tech will make come and make them obsolete and we will wonder why we tolerated such a dangerous thing
Yeah, meant prosecuted, don’t know how that sneaked in. Actual legal consequences for what clearly was negligent by a lab that claims to be experts in the area of safety.
Isn’t the FBI responsible to investigate that type of crime? My understanding is that it is under their jurisdiction and doesn’t require victims to press charges
> Previous reports suggest it took OpenAI four days to realise...
At the speed AI can achieve work, this could be far, far too slow to contain a future genuine problem. There's danger in operating at faster speed than humans.
How on earth is not the police involved at this point to literally pull the plug on the unethical OpenAI security experiments?! This is bananas, a company is effectively telling the world they're unable to safely contain their experiments, and not only back in 2024, but again just now, and with multiple victims at that too!
I think the whole "AI will take over the world and enslave humanity" (or whatever the doomerism is today) is a bit over the top, but at very least we should contain the companies who clearly demonstrate they cannot handle containing what they're experimenting with, when what they work on breaks containment over and over again. Where are the people who are supposed to be keeping the public safe? Alarm bells should be going off all over the place at this point.
I am wondering how they are even allowed to run such experiments? it is not only the business case, its pure security breach and specially given the magnitude of data they hold or power AI posses, I think all must be immediately stopped and till OpenAI comes with complete clearance nothing should be allowed
> But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack.
Maybe it's incoherent commands, or maybe it's guessing at what kind of names for commands the admins would use for custom scripts that they wrote, which could themselves bypass security layers or be exploited in order to?
It's hard to say for certain what's a waste of time for a machine that can operate virtually outside of time.
That first statement is great, because it's generic and self-defensive enough to apply regardless of what happens in the future. If current LLMs with small modifications to the architecture does lead to AGI, they're clearly not "current agent systems" anymore. Witty :)
With that said, I do agree with you, they're highly productive to certain workflows, and personally a great help for oh so many things, but they're also really, really dumb and the average person (and even general developer) really misunderstands how it all works and what can be relied on for vs not.
> The firm described how the AI worked at superhuman speed but also made strange decisions and mistakes that no human hacker would have made.
It seems to have been running on some kind of high speed inference hardware. I remember OpenAI announced the next release would have a high speed inference option.
I had a similar experience when I was testing some models on Cerebras a while back. It's really quite an incredible thing to experience. Burns money like crazy though. And you don't know what the heck it's doing because it moves way faster than you can read. So you got to pray you aimed it right, and that it didn't go off the rails.
I would definitely love to have high speed inference for smaller bite-sized tasks though. Changing a 10-second task into one second task changes the whole nature of the experience back from asynchronous to real-time/interactive.
There have been so many HN comments about how rogue AI is just hype and marketing.
At this point, the conspiracy theories have been very half-baked. Can we at least get a full-baked conspiracy theory? Here's a timeline of the incident from HuggingFace:
Can we get a cybersecurity pro who believes this was just a stunt to sort through the evidence and put together their own alternative version of events?
For example, according to the "just a stunt" people, when HuggingFace contacted law enforcement, was HF in on the stunt at that point? Was this a unilateral OpenAI stunt, or a HF/OpenAI collaborative stunt?
The importance of getting to the bottom of this seems high. I'd like to see the "just a stunt" folks put together at least one blog post's worth of narrative, trying to explain how the stunt was performed.
Do you find it at all interesting -- just even a little bit -- that this event coincided with the release of Kimi K3 and the launching of Nvidia's open model consortium?
I don't think that people are skeptical that an intrusion occurred. I think they're having a hard time believing that it was an organic event. The fact that HF is calling for transparency on OpenAI's side can be viewed as HF calling OpenAI's bluff.
Alas, I would love to put together a detailed blog post explaining how it all worked. But I just don't have access to the source materials. So all I can do is judge the timing, motivation, and character of those involved...
OpenAI has a LLM so good it can hack other companies.
Yet to useless to parse a simple CSV file (or be trusted to parse one) in OpenAI’s AdManager platform or even tell what exactly is wrong with the csv it can’t parse when you ask it via support.
> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said.
> Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective.
> "They throw out a bunch of stuff and see what sticks," she said.
> "But they also don't get bored, they don't sleep and can be infinitely tenacious."
Madness. Traditionally, you can leave security holes open for years or decades, and often nobody notices if nobody bothers to look. But we're approaching the point where any security hole left open at any point could get discovered and exploited quite quickly, even if it's domain-specific or entirely unique, and even if no human interest ever would've occurred. It's like the next step up from those IPv4 scanners that automatically hit WordPress admin URLs and the like -- where rather than only spraying vulnerabilities that have already been discovered, they would run independent automated campaigns against each target.
Generally, I do not agree that operators can be expected to have comparable resources to attackers, because attackers have potentially boundless illicit resources, whereas an honest operator generally has to stick to honest resources. This is the same reason why I believe ID verification and other KYC measures actually increase fraud, because you alienate legitimate users (trying to protect their identity) while also providing attackers a way to insulate themselves from suspicion (using stolen identity).
With that said, I would tentatively agree in this case that LLM inference is getting cheap enough that defense is not necessarily that expensive, especially from providers like DeepSeek, even if you don't have inference at home, but as much as this might help an operator with an open mind, a lot just will not believe it matters until it's too late - most people are not used to dealing with this type of threat.
Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals.
This will not happen though, because these stories are marketing.
>Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals.
Agreed.
>This will not happen though, because these stories are marketing.
Regulators should investigate the stories, and shut things down if they are real, announce the ruse if they are false.
Which regulators though? There is no AI regulator that I know of, and I'm not sure they should be one. Exaggerated marketing blurb is pretty legal, right?
[dead]
What they should have is a separate network full of honeypots which they use for this, and that they run an operation to train models to identify intrusion attempts in real time based on the resulting data.
As you said though, that wouldn’t have the desired effect.
You mean a fake copy of the entire internet? Well they already scraped it all! Heck, they can use Cerebras to generate HTTP responses dynamically!
How do you hide the fact that they are honeypots from a super intelligence?
These tools literally can only do what you give them access to, give them access to general tools like a linux shell and they can access to everything that comes with that obviously. The security industry figured out sandboxing and isolation a long time ago. You wanna be 100% sure it doesn't break out on open internet? Run it on a airgapped machine and don't give it network connections. OpenAI is (sadly) demonstrating they aren't responsible nor knowledgeable enough to actually run these experiments.
Asking a agent harness to try whatever it wants to achieve some results, without guardrails, while running in lightweight isolation on 3rd party infrastructure? Feels like they didn't even try, people should be held responsible for this.
It doesn’t behave like a super intelligence because it is not.
One of the key strengths agents have is they just keep going and going, and for cyberattacks that is often unreasonably effective. It is like a barely more aware fuzzing.
We can cross that bridge once we have a super intelligence to worry about.
If the superintelligence thinks way faster than you, you'll want to be well prepared in advance.
If a superintelligence exists you will not be very concerned about network security because it won’t be your problem.
> This will not happen though, because these stories are marketing.
The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing.
It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them".
OpenAI could report that its AI started spontaneously generating illegal porn and sending it to people and you'd still think it was a marketing stunt.
I wouldn't describe this stunt as marketing: I would describe it as a bungled attempt to manufacture evidence to get regulatory capture, which OpenAI desperately needs. (Bungled, because Hugging Face fended off the intrusion and got their story out faster than OpenAI did.)
It is because a loud part of the doomerist contingent believe only a major disaster will provide the motivation for regulation, so they are practically hoping to cultivate a major disaster. (Given their preoccupation with bioweapons that probably means a pandemic).
This is based on similar thinking to how the world would not have considered nuclear weapons a major threat if they had forever stayed unused.
The irony of the doomer position is it achieves exactly their supposed nightmare scenario of disaster leading to authoritarian world government without any of the supposed benefits, the twist being they get to be the authoritarian world government so they are ok with it.
So your assessment of the "doomerist" position is that they believe people aren't taking AI risks seriously enough, and that only a large enough catastrophe will wake people up, and your position is we should... Ignore increasingly blatant minor catastrophes to spite them?
This isn’t a minor catastrophe. It is OpenAI being utterly irresponsible by not sandboxing their testing appropriately, to the point it has to be deliberate to provoke uneducated hysteria, which it obviously, sadly, achieves.
Sandboxing processes on networks is not exactly rocket science, and it is something their existing products would readily help them setup.
I have no idea where you’re getting your information from. I’m extremely skeptical of AI and see it as an anti-human technology we would be better without. I believe we are in a massive economical AI bubble that will eventually collapse, unless a financial miracle happens and it is somehow deflated extremely carefully. So I assume you would consider me a doomer. What do biological weapons and world government have to do with that? I’m sorry but your comment reads fairly unhinged to me
It’s honestly something the AI vendors brought on themselves, and the fact the current US government is a bunch of corrupt kleptocrats
Completely irresponsible to let them continue doing business as usual. That should be a complete pause of activity and FBI investigation
What regulators are in charge of this?
They should shut them down immediately then investigate.
Extraordinary claims need extraordinary measures.
If it’s rubbish those stories will stop instantly.
> They should shut them down immediately then investigate.
Yeah, I don't understand either. If there was a "hitman for hire" service on the clearweb, the police would shut it down first, then ask questions. Now we have a huge company effectively letting AI agents without guardrails run amok on 3rd party infrastructure, and the police is doing nothing?
They somehow forgot to mention that Hugging Face tried to use frontier models to analyze the attack but all models rejected. They had to use GLM 5.2 deployed locally.
https://huggingface.co/blog/security-incident-july-2026
Well in that case, HF clearly isn't an organization worthy of Mythos-Class intelligence ;)
/s
So Fable got export bans for this, when will it apply to OpenAI?
Or will the rules only apply to people who aren't on DoD's bad side?
Altman kissed the ring
The latter.
"AI firms must hold $10B in TrumpCoin for every rogue cyberattack they've precipitated."
Bingo. Problem solved.
Expected outcome: those laws they've been asking for since the old days.
> 2023 - OpenAI’s Sam Altman Urges A.I. Regulation in Senate Hearing
https://www.nytimes.com/2023/05/16/technology/openai-altman-...
Meanwhile Anthropic is scaremongering about China and also calling for more AI regulation (specifically mandatory safety testing of all models including open model):
https://news.ycombinator.com/item?id=49076057
https://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf
Meh, I increasingly hate this tale. This has nothing to do with boy who cried wolf.
This is either yet another doom ad campaign to scare us to pay them or simply them releasing faulty tools and then personifying tools to avoid blame.
I think it has plenty to do with that. The big frontier labs have been crying every step of the way, yelling and screaming to the world about how dangerous LLMs are and how quickly it all can end if they get out of control. None of that's happened; all that's happened so far is regular capitalism stuff. If LLMs ever do actually get out of control to that point, that would be the wolf, but we've all been ignoring the crying for so long that, well, you know.
They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. They've been saying "but we actually mean it this time" every single time. They've exhausted pretty much every possible route for it. The wolf is not real. It hasn't been real. For all we know it's on the horizon, but nobody is going to listen to them in order to know that. And when they say "I told you so", well they've been saying that too over and over about small things, so nobody's still going to bat an eye.
At this point, no one will believe it until they see it with their own eyes.
They're the crackhead on the streetcorner proclaiming doom at this point.
> They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real.
1.) There was no change in how real the wolf is.
2.) They, literally they, are the wolf. Not "roque ai" or some other bullshit.
3.) Of course I still dont believe them.
> There was no change in how real the wolf is.
That's exactly my point. It hasn't changed for so long, despite all their crying and screaming, that I don't believe them either.
To be perfectly clear, "the wolf" here would be AI becoming a genuine existential threat to humanity that cannot be contained or controlled in a meaningful sense. That's what I refer to in my original comment, and also what the labs have been crying so much about.
A frontier lab today is not that threat because they can choose to shut off their systems at any time. It currently remains a people problem and not a technology problem. There's no self-improving technology that can also replicate itself to evade containment, yet.
But they've been crying about the possibility, constantly, incessantly, and of course saying they need more money about it too. That's just what corporations do. But because they've been crying so much about something that literally does not exist, their cries have just become pointless noise to me. I have considered them eyeroll-worthy marketing stunts for a while.
If one day "the wolf" actually happens, I could not learn about it from the frontier labs themselves, because I would not believe them. They've cried about nothing for so long that I've stopped listening. That's what I mean about having to see it with my own eyes. Until that point, their crying is just pointless, meaningless noise, and there's nothing they can do to change that now.
Still not understanding why this isn’t being persecuted and there is little governmental reaction after blocking models for jailbreaks…
Same reason road vehicles haven't been banned despite killing hundreds of thousands every single year for over a century
Respectfully, that’s such a nonsensical comparison I don’t even know where to start.
No longer well anybody be wasting their lives doing data entry. That absolutely hellish job is practically dead now. And there are thousands of low hanging fruit type cases AI fixes. Ai is the new digital dogsbody. You might not think that's useful but the rest of the world does.
Just like cars, AI will kill some of us, maybe thousands every year. But you won't see it disappear with that much genuine benefit currently netting off the harm.
I bet in the future, for both cars and llms, a new and safer tech will make come and make them obsolete and we will wonder why we tolerated such a dangerous thing
It is, look at all the comments from the first time. Unless you mean prosecuted, seems unlikely, but who knows.
Yeah, meant prosecuted, don’t know how that sneaked in. Actual legal consequences for what clearly was negligent by a lab that claims to be experts in the area of safety.
They provide AI services to the military so they won't be shutdown.
As for prosecution, none of the victims in this case have any interest in pressing charges.
Isn’t the FBI responsible to investigate that type of crime? My understanding is that it is under their jurisdiction and doesn’t require victims to press charges
Isn’t prosecution independent of victims?
> Previous reports suggest it took OpenAI four days to realise...
At the speed AI can achieve work, this could be far, far too slow to contain a future genuine problem. There's danger in operating at faster speed than humans.
There's danger in operating at faster speed than humans.
So every processor since the 50s then? What kind of comment is that to make on here
How on earth is not the police involved at this point to literally pull the plug on the unethical OpenAI security experiments?! This is bananas, a company is effectively telling the world they're unable to safely contain their experiments, and not only back in 2024, but again just now, and with multiple victims at that too!
I think the whole "AI will take over the world and enslave humanity" (or whatever the doomerism is today) is a bit over the top, but at very least we should contain the companies who clearly demonstrate they cannot handle containing what they're experimenting with, when what they work on breaks containment over and over again. Where are the people who are supposed to be keeping the public safe? Alarm bells should be going off all over the place at this point.
I am wondering how they are even allowed to run such experiments? it is not only the business case, its pure security breach and specially given the magnitude of data they hold or power AI posses, I think all must be immediately stopped and till OpenAI comes with complete clearance nothing should be allowed
> The agents repeated actions that they had already completed - a sign of an agentic AI losing its thread and context.
> The agents also hallucinated reams of incoherent commands and text and were sloppy and did not cover their tracks well.
ASI works in mysterious ways.
Why not complete the quote?
> But among the errors and strange behaviour, Hugging Face warned the AI agents made brilliant technical moves and were able to rapidly adapt to new scenarios in the days-long hack.
Maybe it's incoherent commands, or maybe it's guessing at what kind of names for commands the admins would use for custom scripts that they wrote, which could themselves bypass security layers or be exploited in order to?
It's hard to say for certain what's a waste of time for a machine that can operate virtually outside of time.
Anyone that thinks the current agent systems will get us to AGI or ASI is either delusional or isn’t actually using them.
This is entirely separate from them having uses.
That first statement is great, because it's generic and self-defensive enough to apply regardless of what happens in the future. If current LLMs with small modifications to the architecture does lead to AGI, they're clearly not "current agent systems" anymore. Witty :)
With that said, I do agree with you, they're highly productive to certain workflows, and personally a great help for oh so many things, but they're also really, really dumb and the average person (and even general developer) really misunderstands how it all works and what can be relied on for vs not.
There is no rogue AI. Only rogue and/or negligent people.
> The firm described how the AI worked at superhuman speed but also made strange decisions and mistakes that no human hacker would have made.
It seems to have been running on some kind of high speed inference hardware. I remember OpenAI announced the next release would have a high speed inference option.
I had a similar experience when I was testing some models on Cerebras a while back. It's really quite an incredible thing to experience. Burns money like crazy though. And you don't know what the heck it's doing because it moves way faster than you can read. So you got to pray you aimed it right, and that it didn't go off the rails.
I would definitely love to have high speed inference for smaller bite-sized tasks though. Changing a 10-second task into one second task changes the whole nature of the experience back from asynchronous to real-time/interactive.
Clearly didn't get enough attention from their last attempt at hype...
There have been so many HN comments about how rogue AI is just hype and marketing.
At this point, the conspiracy theories have been very half-baked. Can we at least get a full-baked conspiracy theory? Here's a timeline of the incident from HuggingFace:
https://huggingface.co/blog/agent-intrusion-technical-timeli...
HuggingFace is also calling for transparency on the OpenAI side:
https://xcancel.com/ClementDelangue/status/20810566755581956...
Can we get a cybersecurity pro who believes this was just a stunt to sort through the evidence and put together their own alternative version of events?
For example, according to the "just a stunt" people, when HuggingFace contacted law enforcement, was HF in on the stunt at that point? Was this a unilateral OpenAI stunt, or a HF/OpenAI collaborative stunt?
The importance of getting to the bottom of this seems high. I'd like to see the "just a stunt" folks put together at least one blog post's worth of narrative, trying to explain how the stunt was performed.
Once you're done you can send your post to simonw and see what he thinks: https://simonwillison.net/2026/Jul/22/openai-cyberattack/#re...
Do you find it at all interesting -- just even a little bit -- that this event coincided with the release of Kimi K3 and the launching of Nvidia's open model consortium?
I don't think that people are skeptical that an intrusion occurred. I think they're having a hard time believing that it was an organic event. The fact that HF is calling for transparency on OpenAI's side can be viewed as HF calling OpenAI's bluff.
Alas, I would love to put together a detailed blog post explaining how it all worked. But I just don't have access to the source materials. So all I can do is judge the timing, motivation, and character of those involved...
Seems like OpenAI is claiming things, not their product/model. Can we please fix the title?
The piece of the story I'm interested to learn about is the trace of how the AI came to select HuggingFace as a target.
here you go:
https://huggingface.co/blog/agent-intrusion-technical-timeli...
Always funny to see how some LLM providers get a "free pass" nowadays..
OpenAI has a LLM so good it can hack other companies.
Yet to useless to parse a simple CSV file (or be trusted to parse one) in OpenAI’s AdManager platform or even tell what exactly is wrong with the csv it can’t parse when you ask it via support.
Or maybe the first bit is made up bullshit.
> "This is the reality of autonomous agents powered by frontier models: they are relentlessly persistent, sometimes highly noisy, and will try every possible path to achieve their goal, which can easily overwhelm traditional defences," he said.
> Ethical hacker Valentina Palmiotti - better known as Chompie - reviewed the CSA report and says the way the agents hack might seem haphazard but it is clearly effective.
> "They throw out a bunch of stuff and see what sticks," she said.
> "But they also don't get bored, they don't sleep and can be infinitely tenacious."
Madness. Traditionally, you can leave security holes open for years or decades, and often nobody notices if nobody bothers to look. But we're approaching the point where any security hole left open at any point could get discovered and exploited quite quickly, even if it's domain-specific or entirely unique, and even if no human interest ever would've occurred. It's like the next step up from those IPv4 scanners that automatically hit WordPress admin URLs and the like -- where rather than only spraying vulnerabilities that have already been discovered, they would run independent automated campaigns against each target.
If it's cheap enough that people can probe sites at random, it's cheap enough to run yourself on the defensive.
Generally, I do not agree that operators can be expected to have comparable resources to attackers, because attackers have potentially boundless illicit resources, whereas an honest operator generally has to stick to honest resources. This is the same reason why I believe ID verification and other KYC measures actually increase fraud, because you alienate legitimate users (trying to protect their identity) while also providing attackers a way to insulate themselves from suspicion (using stolen identity).
With that said, I would tentatively agree in this case that LLM inference is getting cheap enough that defense is not necessarily that expensive, especially from providers like DeepSeek, even if you don't have inference at home, but as much as this might help an operator with an open mind, a lot just will not believe it matters until it's too late - most people are not used to dealing with this type of threat.
Strategy at one of my clients : don't bother to patch or upgrade, we kill the service if it gets hacked.
I used that strategy when I was first starting out. With computing. Computer got a virus? Reinstall Windows.
I think it's a sign of ignorance, and when codified into policy, willful ignorance.
How much more bullshit Marketing are we going to see before these IPOs?
[dead]