I find that the text that AI produces is sometimes unintelligible without putting _real_ effort into reading it. It's like the AI is becoming so intelligent that it spreads the information around through different sentences in a different way than humans do. I guess if you are a transformer with thousands of tokens of lookahead, why wouldn't you ...
Anyway, I wish the LLMs were trained with a human-text-detector in their training loop. And they can still use a different LLM to produce code and mathematics. However, using the same LLM for both tasks, and we're going to have a hard time understanding what is happening.
I don't buy the "too intelligent to communicate" thing. Feynman was an exceptional communicator. So was Einstein. LLMs are just getting worse at writing, as we continue to aggressively RL them for coding.
that means it's a bad writer. a good writer knows who their reader is and curates their style and complexity to fit. Claude writes like someone who doesn't give a single shit about whether or not you've memorized an initialism it made for some process 10 hours ago. it's the same issue that people have been having with the impenetrability of needless and performative jargon
The problem is that Anthropic and others use the text generated by LLMs also to power the AI's internal thinking loop. This means that text is used in two different ways: one as input to AI, and the other as input to humans.
They are always trying to make the LLMs more intelligent. Guess what kind of output will suffer?
It's more than disorganized - that disorganization hides a bunch of 1) horseshit assertions from nowhere and 2) contradictory statements both given as evidence for the same conclusions.
My entire day is getting praised and apologized to for "holding me to a higher standard" and "pointing out the obvious contradiction." Meanwhile, I'm stuck and can't move forward (because every suggestion is dumb but asserted with the same confidence as the last) and the tokens are draining dry.
When I hear about people converting huge projects to Rust, I'm starting to think that translation is the only thing that they're good at. If you've already entirely specified the problem they can restate the problem without making more than a couple hundred (and getting fewer every day) mistakes.
It's just making it up, and when the things that it is making up are not obvious, it bullshits, it doesn't retrace. Then it builds on that bullshit.
It's not you that's dumb, it's them that are dumb. The people evaluating its output in post-training are also too often deciding that it's they themselves that are misunderstanding, especially when it buries the topic at hand with a bunch of excess verbiage and strange assumptions that they don't have time to independently verify. The need to believe something based on expediency often results in the rationalization that some 3D chess is being played. More like 3 card monte.
> It's not you that's dumb, it's them that are dumb.
I use AI a lot in my workflow and feel like I get a lot out of it. When I first started, I kept reading code & explanations and thinking "man, I just do not understand this explanation, I guess I'm not as good at this as I thought."
Then I started digging with the AI to have it explain stuff to me and I realized, no, it's the AI that's dumb.
Now when I don't understand something that AI generated I understand it's a great sign that it's wrong.
I was curious why someone would need to classify a comment in the first place.
> One thing in this comment caught my mind: the small parenthetical that says “the usual case”. If that were true, it would be important information! However, this comment is generated by the llm model Claude, which has no sense of what counts as the usual case in this domain.
That's from the post referenced early on. I think that's the fundamental misunderstanding of what these tools are supposed to be used for. And I'm just as guilty myself at work. The problem isn't that the LLM generated that text, the problem is that a human didn't read it properly and ask what that meant and fix it before pushing.
> The problem isn't that the LLM generated that text, the problem is that a human didn't read it properly and ask what that meant and fix it before pushing.
The amount of garbage AI generates quickly overwhelms the human. The whole reason for computers to exist is to remove burden from humans. And yet here we are.
The more advanced the effort to detect human vs AI the more advanced the AI will be to write just like human.
I wonder if at some point it will become totally human like, and then what will be the reaction? will you accept it? will it be tuned and targetted just like ads to you so it can be eloquent and influence you?
Yes, my favorite codebase to work in is one full of accurate and long comments that detail that making a function call it never called would cause problems because of the details of a function that gets called 3 layers deeper.
I find that the text that AI produces is sometimes unintelligible without putting _real_ effort into reading it. It's like the AI is becoming so intelligent that it spreads the information around through different sentences in a different way than humans do. I guess if you are a transformer with thousands of tokens of lookahead, why wouldn't you ...
Anyway, I wish the LLMs were trained with a human-text-detector in their training loop. And they can still use a different LLM to produce code and mathematics. However, using the same LLM for both tasks, and we're going to have a hard time understanding what is happening.
I don't buy the "too intelligent to communicate" thing. Feynman was an exceptional communicator. So was Einstein. LLMs are just getting worse at writing, as we continue to aggressively RL them for coding.
that means it's a bad writer. a good writer knows who their reader is and curates their style and complexity to fit. Claude writes like someone who doesn't give a single shit about whether or not you've memorized an initialism it made for some process 10 hours ago. it's the same issue that people have been having with the impenetrability of needless and performative jargon
> that means it's a bad writer.
The problem is that Anthropic and others use the text generated by LLMs also to power the AI's internal thinking loop. This means that text is used in two different ways: one as input to AI, and the other as input to humans.
They are always trying to make the LLMs more intelligent. Guess what kind of output will suffer?
It's more than disorganized - that disorganization hides a bunch of 1) horseshit assertions from nowhere and 2) contradictory statements both given as evidence for the same conclusions.
My entire day is getting praised and apologized to for "holding me to a higher standard" and "pointing out the obvious contradiction." Meanwhile, I'm stuck and can't move forward (because every suggestion is dumb but asserted with the same confidence as the last) and the tokens are draining dry.
When I hear about people converting huge projects to Rust, I'm starting to think that translation is the only thing that they're good at. If you've already entirely specified the problem they can restate the problem without making more than a couple hundred (and getting fewer every day) mistakes.
It's just making it up, and when the things that it is making up are not obvious, it bullshits, it doesn't retrace. Then it builds on that bullshit.
It's not you that's dumb, it's them that are dumb. The people evaluating its output in post-training are also too often deciding that it's they themselves that are misunderstanding, especially when it buries the topic at hand with a bunch of excess verbiage and strange assumptions that they don't have time to independently verify. The need to believe something based on expediency often results in the rationalization that some 3D chess is being played. More like 3 card monte.
> It's not you that's dumb, it's them that are dumb.
I use AI a lot in my workflow and feel like I get a lot out of it. When I first started, I kept reading code & explanations and thinking "man, I just do not understand this explanation, I guess I'm not as good at this as I thought."
Then I started digging with the AI to have it explain stuff to me and I realized, no, it's the AI that's dumb.
Now when I don't understand something that AI generated I understand it's a great sign that it's wrong.
> I find that the text that AI produces is sometimes unintelligible without putting _real_ effort into reading it
I know exactly what you mean and it's fatiguing if you used the tool or whatever for more than a few hours
i can already see the false positives lol, 99% of these ai detectors aren't ai detectors their just specific writing style detectors
I was curious why someone would need to classify a comment in the first place.
> One thing in this comment caught my mind: the small parenthetical that says “the usual case”. If that were true, it would be important information! However, this comment is generated by the llm model Claude, which has no sense of what counts as the usual case in this domain.
That's from the post referenced early on. I think that's the fundamental misunderstanding of what these tools are supposed to be used for. And I'm just as guilty myself at work. The problem isn't that the LLM generated that text, the problem is that a human didn't read it properly and ask what that meant and fix it before pushing.
> The problem isn't that the LLM generated that text, the problem is that a human didn't read it properly and ask what that meant and fix it before pushing.
The amount of garbage AI generates quickly overwhelms the human. The whole reason for computers to exist is to remove burden from humans. And yet here we are.
I ended up vibe-coding a comment linter for my projects that catches the garbage and forces the models to rewrite. Just look at this: https://xcancel.com/dmitriid/status/2097632371650552084#m
my handwritten very detailed 10 lines comment block is %87 robot. and i use broken english with wrong grammer all the time.
you are not the only one, Pangram and GPTZero also think it is written by a bot.
ai detection may be an unsolvable issue.
The more advanced the effort to detect human vs AI the more advanced the AI will be to write just like human. I wonder if at some point it will become totally human like, and then what will be the reaction? will you accept it? will it be tuned and targetted just like ads to you so it can be eloquent and influence you?
https://xkcd.com/810/
Unfortunately my naive heuristic of "if there is a comment it is written by AI" is nearly as accurate.
Quick tip to make AI code better is just delete the comments. Usually they’re CoT that shouldn’t be there.
hahaha 99.9% of the time this is accurate, especially when the comment is accurate and long
Yes, my favorite codebase to work in is one full of accurate and long comments that detail that making a function call it never called would cause problems because of the details of a function that gets called 3 layers deeper.
I think the point should be making sure these AI codes are reviewed by developers, which is hard to do.
Developers write plenty of bad code and comments too, trying to detect AI just feels more like treating the symptom than the actual problem.
This is ultimately a GAN and will converge below the error rate:
https://xkcd.com/810/