15 points | by codyznash 5 hours ago
4 comments
This is great writeup, can we do a cross comparison with "human" experts?? whether models perform better OR worse??
This writeup reads like it was written by Claude, which makes me immediately question its accuracy.
> Each dot is one code sample; bars mark the median. The split between malicious and benign packages, perfect before the prune, was perfect after.
People do not write like this.
I feel like LLMs really highlight the ambiguity and imprecision of the english language in normal use. LLMs are getting really good at guessing what we mean, but it is still a guess.
This is great writeup, can we do a cross comparison with "human" experts?? whether models perform better OR worse??
This writeup reads like it was written by Claude, which makes me immediately question its accuracy.
> Each dot is one code sample; bars mark the median. The split between malicious and benign packages, perfect before the prune, was perfect after.
People do not write like this.
I feel like LLMs really highlight the ambiguity and imprecision of the english language in normal use. LLMs are getting really good at guessing what we mean, but it is still a guess.