Recently I watched this presentation of a kernel developer about the "Mythos' 79 kernel vulnerabilities": https://youtu.be/NnV_cWeoo5Q?t=257
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
That's the weird thing with LLMs, they know just enough about a topic to fool anyone without much knowledge on the subject, seemingly even frontier labs.
It doesn't help that the labs have positioned themselves on the predication that expertise is no longer required.
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
I would believe Anthropic about how Claude reduces attack surfaces if they weren't simultaneously nannying everyone about how they supposedly don't want their models used for cybersecurity because... Reasons. And if you do want that access, you have to verify your identity, and do a bunch of other things, all of which goes against "reducing" the "attack surface". If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
> If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
Golden guns don't keep you safe from silver bullets.
These models are obviously getting used to commit acts which would be criminal if done with intent (https://www.felonybench.com), handing everyone a digital version of a machine gun means everyone catches strays.
Worse: every capability the SOTA models demonstrate is available in an open weight model 3 months later, so we're all catching strays already, and it's only the exact policies you're unhappy about which gave critical infrastructure any prep time.
And recall, absent demonstrations of capabilities (and sometimes even with demonstrations), lots of people respond dismissively to these warnings; before OpenAI had actually hacked into HuggingFace, people thought it was "science fiction" that an AI would find and exploit novel zero-days in its own sandbox.
I’m very frustrated by this, for the record. But they have the atom bomb, and we want to borrow their tech for nuclear power. They don’t trust us. This familiar geopolitical quagmire isn’t resolved on the global scale, so I don’t expect these flawed companies to do it any better.
There is really just one way this goes and its upwards to the peak: which means full formal verification programming languages, OS uni-kernels, small TCB and probably verified HW for the most sensitive cases. Everything else will soon be hackable via 'brute compute' because economy of scale and accessibility of open weight models.
These programs are executed in environments. Which are run on a hardware. Big part of which is hidden software.
And in general you can have 5 boxes verified and be surprised what comes out of them calling each other. Number of boxes is growing and possible interactions is exp. I don't think formal verification solves much.
I'm doing this attack surface tactic with Vix IDE, a terminal text editor that helps novices get comfortable with vi, emacs, git, jj, sql, commands, and more.
The prompt I used was "Goal: rework to reduce the attack surface. Process: move features into feature-flag Rust crates. Output: build a minimal version. Questions: Ask me anything."
This past weekend, Claude Fable/Opus/Sonnet together successfully refactored the whole Vix IDE codebase into many separate Rust crates and Cargo features.
This refactor enables a security-oriented user to pick and choose, and also makes it easier for a security auditor to look at one capability at a time.
If you're curious you can see the commit messages by Claude Sonnet. IMHO they're reasonably good.
Is there not a great irony in over a weekend throwing Fable/Opus/Sonnet at your project, refactoring your whole codebase, and the purpose of that project is to teach people to use vi, git, sql and so on. Why would anyone want to learn that in the current day and age, when as you state it can refactor a project over a weekend fairly autonomously?
Ha! Yes you're right. IMHO the industry is in a limited window of time where software engineers are still using the foundational tooling and not quite yet using AI for autonomous creating. This is especially true in regulated sectors.
If only Claude didn't constantly drop to less capable models whenever it encountered anything vaguely cybersecurity related (despite being part of the Anthropic Cyber Verification Program and having Mythos access, albeit limited).
We're strongly pushing for a move to a mix of Codex and open models at my workplace because of Anthropic's ridiculous nanny-ing.
Yep, gotta really mind the language, sometimes saying something like “when implementing <foo> feature you gotta pay attention to <bar> to avoid <baz> attack” is enough to trigger it, along with a cache miss of 300k tokens in the downgraded request to rub it in. Frustrating.
These types of flip flop posts are the new security theater. One week Anthropic will post a doom and gloom about how open weight models running a fraction of a token per second on your Grandma's popped Android TV is a national security threat and the next week they'll tell you how awesome their 200+ billion parameter model is at doing something they don't actually allow you to do because they gatekeep all the normies. I really do hope history remembers them correctly, but I'm sure they'll be attributed as being a significant and load bearing linchpin during the new world order of LLMs.
Also in the CVP and kept tripping cyber a couple days ago due to an MCP server that had the ability to connect to remote devices. Maybe it thought I was trying to make a C&C server. But you literally have to start the MCP first, have it generate a key, give the key to the remote device and then it'll generate its key, then give that key back to the MCP server before they can connect.
React is actually pretty good for any kind of reactive interface, and it's small. Something under 3k lines of code, and it's well battle tested at this point.
You might have a dislike of react and that's fine, but it's not adding any significant attack surface to a project.
Just the ReactFiberWorkLoop.js has 5k+ lines of code. React is one or two orders of magnitude bigger than you are thinking. And that's not counting Nextjs.
> React is actually pretty good for any kind of reactive interface,
It’s a different interface and the axiomatic layers are not the same.
React expects a tree of nodes that will map to the UI and build on that to add reactivity. And that’s what the major GUI platforms do offer (QT, the web dom, Android, iOS,…).
The terminal is a grid of characters modfied by the cursor concept. It is also modal (although most TUI uses one mode). What the cursor can do, and how the grid is rendered is terminal dependent (even though most current ones try to follow xterm-256color).
Using React on top of that is a technical bad idea.
This sounds so arrogant, lol. I know it actually means that they red team the frontier AI models (which is a reasonable claim), but it looks like they are claiming to be the best red team in the world
I literally included it. more broadly: dont install any so-called AI/LLM "black box" agent anywhere if you want to minimize attack surface and risk. Claude might be the most notoriously foolish but the same security-sloppy design decisions are likely present in the other similar codebases. And the line between reckless naivete and intentional maliciousness is conveniently blurry.
These guys are now the biggest slowdown to fixing issues with their bullshit cyber detection. They drop a model, let the model find security issues in your code, then when you want to fix them you need to play a game of charades so it doesn't realize it's creating a validation and then a fix for the issue because you might be hacking yourself. Meanwhile attackers are using some ablated deepseek or what have you.
It's frustrating. Claude thinks it finds a way to hop from one client to another, but any attempt to get it to demonstrate it isn't relying on something that's already prevented runs into "safety" concerns and gets blocked.
I'm not that hot on the idea of open models, but having something dangled in front of you but suddenly unable to verify is not a good feeling.
With the right amount of coaxing you can sometimes get it to reproduce a PoC with a "TDD first" approach to fixing, but often now that gets blocked too.
I'm not opposed to open models at all, I'm just slightly sceptical on how much some people push their use compared to the value we get out of a dirt cheap subscription.
Propriety models are clearly subsidised by enterprise API contracts, so it makes sense to milk that for all its worth, provided one steers clear of any lock in so the option to switch is always there.
Jumping to spending time managing and self-hosting open models is a nice ideal but I think it's a distraction to jump on that train without proven benefits given the value that other plans deliver.
I don't mean I'm against the idea of open models, they were especially instrumental in Anthropic backing down from the "Fable only initially available 2 weeks" plan, which got extended up until Fable was over-taken by Opus in general usefulness.
It's my view that any company sized appropriately for Teams plan should be milking it for all its worth while it still exists, and it is a waste to lose that opportunity by jumping on open models too early.
If you're a larger enterprise then it would be a very different story, you'd already be paying more forced onto API pricing and would likely have the capacity to have a team look at open models without impacting delivery.
there is a middle ground between proprietary models and self-hosting: Fireworks, et al. Popular models like DeepSeek have a couple dozen providers on openrouter.
On the other hand this sounds like a job guarantee, though. And not only for the moment, as it is not the capability of the model which is insufficient.
So far I can always get around it by starting fresh sessions on a similar problem and without ever mentioning security issues, but why do I have to play this little theater with a tool we pay to use.
Recently I watched this presentation of a kernel developer about the "Mythos' 79 kernel vulnerabilities": https://youtu.be/NnV_cWeoo5Q?t=257
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
That's the weird thing with LLMs, they know just enough about a topic to fool anyone without much knowledge on the subject, seemingly even frontier labs.
It doesn't help that the labs have positioned themselves on the predication that expertise is no longer required.
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
I would believe Anthropic about how Claude reduces attack surfaces if they weren't simultaneously nannying everyone about how they supposedly don't want their models used for cybersecurity because... Reasons. And if you do want that access, you have to verify your identity, and do a bunch of other things, all of which goes against "reducing" the "attack surface". If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
> If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
Golden guns don't keep you safe from silver bullets.
These models are obviously getting used to commit acts which would be criminal if done with intent (https://www.felonybench.com), handing everyone a digital version of a machine gun means everyone catches strays.
Worse: every capability the SOTA models demonstrate is available in an open weight model 3 months later, so we're all catching strays already, and it's only the exact policies you're unhappy about which gave critical infrastructure any prep time.
And recall, absent demonstrations of capabilities (and sometimes even with demonstrations), lots of people respond dismissively to these warnings; before OpenAI had actually hacked into HuggingFace, people thought it was "science fiction" that an AI would find and exploit novel zero-days in its own sandbox.
I’m very frustrated by this, for the record. But they have the atom bomb, and we want to borrow their tech for nuclear power. They don’t trust us. This familiar geopolitical quagmire isn’t resolved on the global scale, so I don’t expect these flawed companies to do it any better.
There is really just one way this goes and its upwards to the peak: which means full formal verification programming languages, OS uni-kernels, small TCB and probably verified HW for the most sensitive cases. Everything else will soon be hackable via 'brute compute' because economy of scale and accessibility of open weight models.
These programs are executed in environments. Which are run on a hardware. Big part of which is hidden software.
And in general you can have 5 boxes verified and be surprised what comes out of them calling each other. Number of boxes is growing and possible interactions is exp. I don't think formal verification solves much.
It can't be perfect, but don't let the perfect be the enemy of the good.
I'm doing this attack surface tactic with Vix IDE, a terminal text editor that helps novices get comfortable with vi, emacs, git, jj, sql, commands, and more.
The prompt I used was "Goal: rework to reduce the attack surface. Process: move features into feature-flag Rust crates. Output: build a minimal version. Questions: Ask me anything."
https://github.com/vixide/vix
This past weekend, Claude Fable/Opus/Sonnet together successfully refactored the whole Vix IDE codebase into many separate Rust crates and Cargo features.
This refactor enables a security-oriented user to pick and choose, and also makes it easier for a security auditor to look at one capability at a time.
If you're curious you can see the commit messages by Claude Sonnet. IMHO they're reasonably good.
Is there not a great irony in over a weekend throwing Fable/Opus/Sonnet at your project, refactoring your whole codebase, and the purpose of that project is to teach people to use vi, git, sql and so on. Why would anyone want to learn that in the current day and age, when as you state it can refactor a project over a weekend fairly autonomously?
Ha! Yes you're right. IMHO the industry is in a limited window of time where software engineers are still using the foundational tooling and not quite yet using AI for autonomous creating. This is especially true in regulated sectors.
I'd really expect a company like Anthropic to proof-read their posts. I found at least 4 typos in this.
Maybe they're signalling that the post was written by a human
That's what Claude wants you to think.
Ironically, I'm sure Claude could have spotted all four.
> leaving a features behavior undefined
Missing an apostrophe? A human wrote this! Love to see it.
If only Claude didn't constantly drop to less capable models whenever it encountered anything vaguely cybersecurity related (despite being part of the Anthropic Cyber Verification Program and having Mythos access, albeit limited).
We're strongly pushing for a move to a mix of Codex and open models at my workplace because of Anthropic's ridiculous nanny-ing.
Yep, gotta really mind the language, sometimes saying something like “when implementing <foo> feature you gotta pay attention to <bar> to avoid <baz> attack” is enough to trigger it, along with a cache miss of 300k tokens in the downgraded request to rub it in. Frustrating.
• Mythos 5.1's safeguards stopped the response above · continuing once with that noted
These types of flip flop posts are the new security theater. One week Anthropic will post a doom and gloom about how open weight models running a fraction of a token per second on your Grandma's popped Android TV is a national security threat and the next week they'll tell you how awesome their 200+ billion parameter model is at doing something they don't actually allow you to do because they gatekeep all the normies. I really do hope history remembers them correctly, but I'm sure they'll be attributed as being a significant and load bearing linchpin during the new world order of LLMs.
> their 200+ billion parameter model
It's a 10T BTW.
Also in the CVP and kept tripping cyber a couple days ago due to an MCP server that had the ability to connect to remote devices. Maybe it thought I was trying to make a C&C server. But you literally have to start the MCP first, have it generate a key, give the key to the remote device and then it'll generate its key, then give that key back to the MCP server before they can connect.
The irony of a company using Nextjs for a chat app and React for a terminal UI talking about reducing attack surface.
React is actually pretty good for any kind of reactive interface, and it's small. Something under 3k lines of code, and it's well battle tested at this point.
You might have a dislike of react and that's fine, but it's not adding any significant attack surface to a project.
Nextjs on the other hand...
You should add the entire JS runtime you have to ship along React to that attack surface.
Wait until you hear about the attack surface you're exposed to when running Linux!
> You might have a dislike of react and that's fine
That's mighty white of you, it's a dirty job but someone's got to tell others what experiences and feelings are fine
> That's mighty white of you
The fuck kind of racism is this? Calling people white for using basic rhetoric?
Just the ReactFiberWorkLoop.js has 5k+ lines of code. React is one or two orders of magnitude bigger than you are thinking. And that's not counting Nextjs.
No that's including all the comments, types, dev only checks etc. I'm talking about what you get when using the production build.
I had a quick look around and it's actually ~1000 lines apparently, in React 19.
We're only 10 months out from React2Shell. That is not nearly enough time to say "battle-tested".
> React is actually pretty good for any kind of reactive interface,
It’s a different interface and the axiomatic layers are not the same.
React expects a tree of nodes that will map to the UI and build on that to add reactivity. And that’s what the major GUI platforms do offer (QT, the web dom, Android, iOS,…).
The terminal is a grid of characters modfied by the cursor concept. It is also modal (although most TUI uses one mode). What the cursor can do, and how the grid is rendered is terminal dependent (even though most current ones try to follow xterm-256color).
Using React on top of that is a technical bad idea.
Nice article! Thanks for sharing.
> Team: Frontier Red Team
This sounds so arrogant, lol. I know it actually means that they red team the frontier AI models (which is a reasonable claim), but it looks like they are claiming to be the best red team in the world
Considering they have access to infinite free tokens they should be the best.
base rule: dont use Claude. never run it or allow it to be installed anywhere
What's you're recommended alternative?
I literally included it. more broadly: dont install any so-called AI/LLM "black box" agent anywhere if you want to minimize attack surface and risk. Claude might be the most notoriously foolish but the same security-sloppy design decisions are likely present in the other similar codebases. And the line between reckless naivete and intentional maliciousness is conveniently blurry.
These guys are now the biggest slowdown to fixing issues with their bullshit cyber detection. They drop a model, let the model find security issues in your code, then when you want to fix them you need to play a game of charades so it doesn't realize it's creating a validation and then a fix for the issue because you might be hacking yourself. Meanwhile attackers are using some ablated deepseek or what have you.
It's frustrating. Claude thinks it finds a way to hop from one client to another, but any attempt to get it to demonstrate it isn't relying on something that's already prevented runs into "safety" concerns and gets blocked.
I'm not that hot on the idea of open models, but having something dangled in front of you but suddenly unable to verify is not a good feeling.
With the right amount of coaxing you can sometimes get it to reproduce a PoC with a "TDD first" approach to fixing, but often now that gets blocked too.
Why wouldn't everyone be hot on open models? Security implications? First time I have heard someone lightly opposing the idea of fully open models.
I'm not opposed to open models at all, I'm just slightly sceptical on how much some people push their use compared to the value we get out of a dirt cheap subscription.
Propriety models are clearly subsidised by enterprise API contracts, so it makes sense to milk that for all its worth, provided one steers clear of any lock in so the option to switch is always there.
Jumping to spending time managing and self-hosting open models is a nice ideal but I think it's a distraction to jump on that train without proven benefits given the value that other plans deliver.
I don't mean I'm against the idea of open models, they were especially instrumental in Anthropic backing down from the "Fable only initially available 2 weeks" plan, which got extended up until Fable was over-taken by Opus in general usefulness.
It's my view that any company sized appropriately for Teams plan should be milking it for all its worth while it still exists, and it is a waste to lose that opportunity by jumping on open models too early.
If you're a larger enterprise then it would be a very different story, you'd already be paying more forced onto API pricing and would likely have the capacity to have a team look at open models without impacting delivery.
there is a middle ground between proprietary models and self-hosting: Fireworks, et al. Popular models like DeepSeek have a couple dozen providers on openrouter.
There's a lot of lobbying against open models on the grounds of a) China and b) open models mean we won't be able to prevent a hard takeoff.
On the other hand this sounds like a job guarantee, though. And not only for the moment, as it is not the capability of the model which is insufficient.
So far I can always get around it by starting fresh sessions on a similar problem and without ever mentioning security issues, but why do I have to play this little theater with a tool we pay to use.
> Meanwhile attackers are using some ablated deepseek or what have you.
I think the hackers are just using "not Claude".
In the kingdom of vibe coderz the engineer is king!