For each likely AI-generated target, we search human controls from the same repository and same programming language, controlling for project-level practices … We also apply size matching: a control must be within 20% lines of code of the AI target
And then from the start of their results and analysis:
Comparing the lines of code (LOC) between AI-generated and human-written files, we observed no statistically significant differences
Here I would expect some sort of caveat that this result is a verification that their size-matching was effective in removing differences in lines of code between the target and control. However, there is no such thing, indicating to me that the authors are unaware that what they are detecting are the effects of their own methods. This leads me to strongly discount their stated insight regarding structural differences, and frankly the rest of their paper too.
Near the end of their methods:
And then from the start of their results and analysis: Here I would expect some sort of caveat that this result is a verification that their size-matching was effective in removing differences in lines of code between the target and control. However, there is no such thing, indicating to me that the authors are unaware that what they are detecting are the effects of their own methods. This leads me to strongly discount their stated insight regarding structural differences, and frankly the rest of their paper too.