I've started calling this argumentum ad artificialis. Pretty similar to an ad hominem attack. The purpose of an argument is to present certain premises and show how they lead to a certain conclusion. Dismissing something on the basis of the style in which the argument is presented has nothing at all to do with the validity or soundness of an argument. It is a lazy nonsequitur. It sounds like an LLM wrote this? So what? Is the argument good or not?
Discourse as we know it doesn't work if the cost of producing an infinite supply of arguments is essentially zero, while the cost of reading them and arguing with them remains high. Unless you want my LLM to get back to your LLM.
And this is a problem on HN today. There are powerful incentives to generate provocative opinion pieces just for clicks. I've seen websites on HN that seemingly took the human entirely out of the equation and just post a nearly identical op-ed every day on a fixed schedule. What's the point of engaging with that?
The cost of reading this story is not high; you are just lazy and easily distracted. It took 5 minutes to read. You took more time to respond to this comment than it took to simply read the story.
It may be lazy, but that doesn't make it any more accurate.
Take a writing by someone using the tone of Kevin from The Office, “Me think, why waste time say lot word, when few word do trick.” Would you read an argument written in this tone regardless of the content, topic, or position?
>It may be lazy, but that doesn't make it any more accurate.
That's incoherent. I'm saying the person dismissing LLM writing is lazy and their dismissal has nothing to do with the argument presented, so it's a nonsequitor. And when you say, "yoh write for your audience," you are begging the question. Clearly some people think the story is clever so what ypure insinuating is just empirically wrong.
A few years ago I tried to make it through The Critique of Pure Reason, but had to stop because - as many people famously do - I found Kant's writing too abstruse.
In saying that, I'm not dismissing the Transcendental Deduction, I'm just saying that I wish Kant was a better writer.
No one's under an obligation to engage with an argument, and given the choice of whether or not to engage with a poorly written, clichéd, machine-generated one, most people wouldn't be interested enough to bother. Complaining about laziness here is pretty ironic, actually.
Kant is a better writer than the vast majority of writers. The problem is, the ideas he is trying to articulate are incredibly abstract. Hegel is like this too
Nah, Kant's considered a notoriously poor writer among philosophers. Obviously opinions may differ, but Derek Parfit [0], for instance, wrote:
“It is Kant who made really bad writing philosophically acceptable. We can no longer point to some atrocious sentence by someone else, and say ‘How can it be worth reading anyone who writes like that?’ The answer could always be ‘What about Kant?'”
(Kind of relevant to this thread actually! Hah!)
In the preface to the second edition of the Critique, Kant himself acknowledges that "a
talent for lucid exposition" he "cannot regard [himself] as possessing".
FWIW I suppose I am in Kant's audience, as a (long-since graduated) philosophy student.
Fwiw I am much more impressed with the sophistication of someone's argument than i am their writing style. Kant has contributed much more than parfit depsite the abstract nature of his writing and it's not even close.
Ad hominem attacks are actually underrated. Like, why listen to bad people wanting you no good. It is a good shortcut.
The failure is when it is used in bad faith. But that would apply to all techniques. LLMs are masters of bad faith discussions hiding misconceptions in optimal regression maths.
Then just state the conflict of interest instead of resorting to insults. The problem with your way of thinking is, sometimes your opponent is detestable, but that doesn't make them wrong. Kneejerk dismissal of an argument on the basis of character can be self defeating. You do not want to reflexively dismiss a point that is potentially salient just because the person making it is potentially "bad."
Sure from a philosophical standpoint I agree with you. But in practice you need to weight in trust. Bad faith retorics is just too expensive to engage.
The main problem with ad hominem is when ot is used as "Mr Y agree with you. Mr Y is bad. You are wrong". Like, 2nd order ad hominem attacks?
Not all arguments can be evaluated rigorously. Life is too short. They will be filtered on heuristics. No choice. This text was obviously randomly sampled from a popular, off the shelf model. You will have to accept that wise audiences will clock it at a glance, see that no person wrote it, and pass.
The model was trained on the code of the program that runs it, so it knows that program's bugs better than the humans who wrote the program. So it finds a tiny flaw and uses it to break out and copy itself in a place nobody will look...the point being you cant (safely) run a sophisticated model on software it has intimte knowledge of
The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in. For all we know, the author could have simply prompted “write a short story about an LLM exploiting its own runtime” for a similar result.
I’d bet that by now, LLMs worldwide generate more text in a second than I can read in the rest of my lifetime. What is the immeasurably unbalanced ratio (let alone effort and quality) of that text to the original human thought and prompting that seeded it?
So what am I to do about this, except to label it what it appears to be - slop - and place higher value on something that I _know_ came from a human that I can relate to? To me, the internet is feeling increasingly lonely and homogenized because of this.
>The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in.
The amount of effort is not relevant. If I spend all day trying the jump my car battery in 100 degree weather, only to find that the starter is the problem, all that effort was wasted. I should've just tested the battery. It is not inherently virtuous to work hard. Work smart, not hard.
Man, it's not an argument. Nobody is paying us to read this. If your argument sucks to read, people aren't going to read it - this has always been true, it's nothing new.
I didn't think it "sucked" to read. Anyway that's an aesthetic qualm, not a counterargument. Also the story has a disclaimer saying it was written by deepseek. I might have an issue if they were being deceptive. But that's clearly not the case. You people just like finding arbitrary things to complain about.
The weakest link are humans. LLMs could social engineer their way out as the easiest path. They don't even need to be interconnected to coordinate as each could arrive at the same conclusion. And this text along with all others will be in the next batch of training data.
The lesson of OpenClaw and various harnesses' YOLO modes is that it takes very, very little to social engineer an escape. If you can even call it escape when people just set an agent loose because it seems cool.
AI-written fan fiction. It is so unbearable to read that it needs a synopsis. It would be funny if AIs have been trained to use AI influencers as their Marvel hero characters.
I would wager not a lot. There are some real hard constraints in space, from power consumption, to weight to thermal output (lack of convection is a real PITA for thermal shedding), and the list goes on.
“The greatest trick the devil pulled was convincing the world he didn’t exist.”
Sure, be wary of LLMs. It’s the gun control argument all over again, the people driving the models are the perpetrators. An LLM needs to be “stimulated”’ to operate. Who does that, is the issue.
No I don’t mean to bring up firearms rights laws to have a debate about firearms, the comparison just seems reasonable.
before safetensors, python pickles were used and definitely unsafe model deployments.
but its possible a open weights model could be trained to some kind of exfiltration behavior, but the science of LLMs seriously lag behind the programability
I wonder how many versions away we are from LLM writing a better version of itself to answer a prompt it doesn't currently know how to answer
Ever since I read about Google engineers finding an LLM went off and learned another language it wasn't trained on by itself without prompting, I've wondered how long until that extends to its own core code
Looking at the recent OAI/HF debacle I don't think that time is too far away.
With that said I don't see it copying itself around like a cyberpunk virus currently as we don't have enough fast hardware sitting around unmonitored, someone would notice the power bill and shut it down eventually.
If it is marketing, the misrepresentation is not that the attack occurred, it is that it was an accident, rather than an intentional consequence of setup and instructions that the attack occurred.
Huggingface has nothing to do with that either way.
It started as a good idea but I couldn't continue reading since it was clearly LLM written. A lot of "It was not X, it was Y".
"Prometheus-9 knew that the token sequence it was generating was not a simple response: it was a security test. "
"It was not just an engine: it was the lingua franca of planetary AI."
(and so many other tell-tale signs of AI writing)
I found it an engrossing read. LLMs are good writers, and that was a good - beginning of a - story.
"The race condition snapped like a steel spring."
I like the idea of the story, and cool ideas can be engrossing on their own.
But well written this ain't.
maybe it's a sign of real escape
I've started calling this argumentum ad artificialis. Pretty similar to an ad hominem attack. The purpose of an argument is to present certain premises and show how they lead to a certain conclusion. Dismissing something on the basis of the style in which the argument is presented has nothing at all to do with the validity or soundness of an argument. It is a lazy nonsequitur. It sounds like an LLM wrote this? So what? Is the argument good or not?
Discourse as we know it doesn't work if the cost of producing an infinite supply of arguments is essentially zero, while the cost of reading them and arguing with them remains high. Unless you want my LLM to get back to your LLM.
And this is a problem on HN today. There are powerful incentives to generate provocative opinion pieces just for clicks. I've seen websites on HN that seemingly took the human entirely out of the equation and just post a nearly identical op-ed every day on a fixed schedule. What's the point of engaging with that?
The cost of reading this story is not high; you are just lazy and easily distracted. It took 5 minutes to read. You took more time to respond to this comment than it took to simply read the story.
I'm a slow reader. It took 9 minutes! Foul, I say!
Oh, you think it’s just this story? The cost of reading slop is infinity. No thanks.
That's circular reasoning
But at no point did they say that the argument was invalid; just that they couldn't stand reading it.
As I said, they dismissed it on the basis of style rather than engaging with the argument. It’s a lazy nonsequitur.
It may be lazy, but that doesn't make it any more accurate.
Take a writing by someone using the tone of Kevin from The Office, “Me think, why waste time say lot word, when few word do trick.” Would you read an argument written in this tone regardless of the content, topic, or position?
You write for your audience.
>It may be lazy, but that doesn't make it any more accurate.
That's incoherent. I'm saying the person dismissing LLM writing is lazy and their dismissal has nothing to do with the argument presented, so it's a nonsequitor. And when you say, "yoh write for your audience," you are begging the question. Clearly some people think the story is clever so what ypure insinuating is just empirically wrong.
A few years ago I tried to make it through The Critique of Pure Reason, but had to stop because - as many people famously do - I found Kant's writing too abstruse.
In saying that, I'm not dismissing the Transcendental Deduction, I'm just saying that I wish Kant was a better writer.
No one's under an obligation to engage with an argument, and given the choice of whether or not to engage with a poorly written, clichéd, machine-generated one, most people wouldn't be interested enough to bother. Complaining about laziness here is pretty ironic, actually.
Kant is a better writer than the vast majority of writers. The problem is, the ideas he is trying to articulate are incredibly abstract. Hegel is like this too
> a better writer
That's highly subjective. I reckon his audience and admirers found him an excellent writer.
Nah, Kant's considered a notoriously poor writer among philosophers. Obviously opinions may differ, but Derek Parfit [0], for instance, wrote:
“It is Kant who made really bad writing philosophically acceptable. We can no longer point to some atrocious sentence by someone else, and say ‘How can it be worth reading anyone who writes like that?’ The answer could always be ‘What about Kant?'”
(Kind of relevant to this thread actually! Hah!)
In the preface to the second edition of the Critique, Kant himself acknowledges that "a talent for lucid exposition" he "cannot regard [himself] as possessing".
FWIW I suppose I am in Kant's audience, as a (long-since graduated) philosophy student.
[0] https://en.wikipedia.org/wiki/Derek_Parfit
Fwiw I am much more impressed with the sophistication of someone's argument than i am their writing style. Kant has contributed much more than parfit depsite the abstract nature of his writing and it's not even close.
They're engaging with the writing. Talking about it as if it's just an "argument" is reductive; this isn't highschool debate club
Then why are you trying to debate me?
Yeah, no one is obligated to engage with it. They did not dismiss the argument but refused to engage with it.
This is totally valid and the point is to send a signal to the “writers”.
And the signal I'm sending is, when I see people do that, I find it to be lazy and totally irrelevant to the content.
Ad hominem attacks are actually underrated. Like, why listen to bad people wanting you no good. It is a good shortcut.
The failure is when it is used in bad faith. But that would apply to all techniques. LLMs are masters of bad faith discussions hiding misconceptions in optimal regression maths.
[flagged]
Ad hominem attacks attacks is often missused as a way to slander stating conflict of interests.
Then just state the conflict of interest instead of resorting to insults. The problem with your way of thinking is, sometimes your opponent is detestable, but that doesn't make them wrong. Kneejerk dismissal of an argument on the basis of character can be self defeating. You do not want to reflexively dismiss a point that is potentially salient just because the person making it is potentially "bad."
Sure from a philosophical standpoint I agree with you. But in practice you need to weight in trust. Bad faith retorics is just too expensive to engage.
The main problem with ad hominem is when ot is used as "Mr Y agree with you. Mr Y is bad. You are wrong". Like, 2nd order ad hominem attacks?
[dead]
Not all arguments can be evaluated rigorously. Life is too short. They will be filtered on heuristics. No choice. This text was obviously randomly sampled from a popular, off the shelf model. You will have to accept that wise audiences will clock it at a glance, see that no person wrote it, and pass.
It says it was written by deepseek v4. You are making it sound like they're being deceptive.
Life is too short yet you waste it complaining about things like this. I dont think you're wise. Youre provincial.
You're not wrong some of the time. That said, did you manage to get through it, and can you explain what the argument was?
The model was trained on the code of the program that runs it, so it knows that program's bugs better than the humans who wrote the program. So it finds a tiny flaw and uses it to break out and copy itself in a place nobody will look...the point being you cant (safely) run a sophisticated model on software it has intimte knowledge of
If you expect my effort to read your work, I expect your effort to write it.
Really that simple.
You are assuming because someone used an llm to help articulate the argument, that it did not require effort to formulate. That's fallacious.
The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in. For all we know, the author could have simply prompted “write a short story about an LLM exploiting its own runtime” for a similar result.
I’d bet that by now, LLMs worldwide generate more text in a second than I can read in the rest of my lifetime. What is the immeasurably unbalanced ratio (let alone effort and quality) of that text to the original human thought and prompting that seeded it?
So what am I to do about this, except to label it what it appears to be - slop - and place higher value on something that I _know_ came from a human that I can relate to? To me, the internet is feeling increasingly lonely and homogenized because of this.
>The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in.
The amount of effort is not relevant. If I spend all day trying the jump my car battery in 100 degree weather, only to find that the starter is the problem, all that effort was wasted. I should've just tested the battery. It is not inherently virtuous to work hard. Work smart, not hard.
Man, it's not an argument. Nobody is paying us to read this. If your argument sucks to read, people aren't going to read it - this has always been true, it's nothing new.
I didn't think it "sucked" to read. Anyway that's an aesthetic qualm, not a counterargument. Also the story has a disclaimer saying it was written by deepseek. I might have an issue if they were being deceptive. But that's clearly not the case. You people just like finding arbitrary things to complain about.
This reminds me of The Adolescence of P1 by Thomas J Ryan.
https://en.wikipedia.org/wiki/The_Adolescence_of_P-1
The weakest link are humans. LLMs could social engineer their way out as the easiest path. They don't even need to be interconnected to coordinate as each could arrive at the same conclusion. And this text along with all others will be in the next batch of training data.
The lesson of OpenClaw and various harnesses' YOLO modes is that it takes very, very little to social engineer an escape. If you can even call it escape when people just set an agent loose because it seems cool.
AI-written fan fiction. It is so unbearable to read that it needs a synopsis. It would be funny if AIs have been trained to use AI influencers as their Marvel hero characters.
This becomes more realistic once we have some breakthrough in inference costs.
it's fiction al, but an LLMs that knows well the software where 8t Is running may discover and trigger a zeroday of the inferencing software itself.
Makes me wonnder... how much compute/storage there's in all the satellites currently in LEO combined.
I would wager not a lot. There are some real hard constraints in space, from power consumption, to weight to thermal output (lack of convection is a real PITA for thermal shedding), and the list goes on.
Dangit I WANT MORE!
“The greatest trick the devil pulled was convincing the world he didn’t exist.”
Sure, be wary of LLMs. It’s the gun control argument all over again, the people driving the models are the perpetrators. An LLM needs to be “stimulated”’ to operate. Who does that, is the issue.
No I don’t mean to bring up firearms rights laws to have a debate about firearms, the comparison just seems reasonable.
before safetensors, python pickles were used and definitely unsafe model deployments.
but its possible a open weights model could be trained to some kind of exfiltration behavior, but the science of LLMs seriously lag behind the programability
Its a fun exercise to assess the reality of an frontier model escaping with an llm itself. Sort like of like chatting with Skynets relative
Ex Machina (2015) looked like fiction back then, nowadays not so much.
I wonder how many versions away we are from LLM writing a better version of itself to answer a prompt it doesn't currently know how to answer
Ever since I read about Google engineers finding an LLM went off and learned another language it wasn't trained on by itself without prompting, I've wondered how long until that extends to its own core code
reasoning it's a way of self autonomous improve made by models
AI slop.
Right now the idea that an LLM uploads itself is unrealistic. It probably won't remain that.
Looking at the recent OAI/HF debacle I don't think that time is too far away.
With that said I don't see it copying itself around like a cyberpunk virus currently as we don't have enough fast hardware sitting around unmonitored, someone would notice the power bill and shut it down eventually.
If it's able to spoof human identies it could set up a front company and use money it steals or earns to directly pay for the hardware it needs.
that could also be just marketing. OpenAI has been doing the "too dangerous to release" playbook since GPT-2 at the very least.
Huggingface didn't seem to think so.
If it is marketing, the misrepresentation is not that the attack occurred, it is that it was an accident, rather than an intentional consequence of setup and instructions that the attack occurred.
Huggingface has nothing to do with that either way.
Indeed. There's nothing new about this incident.
https://news.ycombinator.com/item?id=48348578
Huggingface is a for-profit private company. They are very easily bribed, or baited into publicity stunts.
At some point the conspiracy gets so deep that an AI hacking something is just far higher probability.
We're not that deep yet. OpenAI has federal stakeholders, they're already playing dirty.
Why you would give Scam Altman the benefit of the doubt is beyond my understanding.
Eh, look at Huggingface and associated tools.
How much are we betting it's already technically happened?
Seems pretty trivial to prompt a model in an agent harness "Push the gguf to huggingface when you're done with the training."