Provenance matters. An LLM cannot certify a Developer Certificate of Origin (https://en.wikipedia.org/wiki/Developer_Certificate_of_Origi...) and a developer of integrity cannot certify the DCO for code emitted by an LLM, certainly not an LLM trained on code of unknown provenance. It is well-known that LLMs sometimes produce verbatim or near-verbatim copies of their training data, most of which cannot be used without attribution (and may have more onerous license requirements). It is also well-known that they don't "understand" semantics: they never make changes for the right reason.
For a large LLM I think the science in the end will demonstrate that verbatim reproduction is not coming from verbatim recording, as the structure really isn’t setup that way in the models under question here.
This is similar to the ruling by Alsup in the Anthropic books case that the training is “exceedingly transformative”. I would expect a reinterpretation or disagreement on this front from another case to be both problematic and likely eventually overturned.
I don’t actually think provenance is a problem on the axis you suggest if Alsups ruling holds. That said I don’t think that’s the only copyright issue afoot - the copyright office writing on copyrightability of outputs from the machine essentially requires that the output fails the Feist tests for human copyrightability.
More interesting to me is how this might realign the notion of copyrightability of human works further as time goes on, moving from every trivial derivative bit of trash potentially being copyrightable to some stronger notion of, to follow the feist test, independence and creativity. Further it raises a fairly immediate question in an open source setting if many individual small patch contributions themselves actually even pass those tests - they may well not, although the general guidance is to set the bar low - but is a typo fix either? There is so far to go on this rabbit hole.
I'd be fine with that if that was the way copyright law had been applied to humans for the last 30+ years but it's not. Look into the OP's link on clean room reverse engineering, I come from an RE background and people are terrified of accidentally absorbing "tainted" information through extremely indirect means because it can potentially used against them in court.
I swear the ML community is able to rapidly change their mind as to whether "training" an AI is comparable to human cognition based on whichever one is beneficial to them at any given instant.
So if you can get an LLM to produce music lyrics, for example, or sections from a book, those would be considered novel works given the encoding as well?
"an LLM" could imply an LLM of any size, for sufficiently small or focused training sets an LLM may not be transformative. There is some scale at which the volume and diversity of training data and intricacy of abstraction moves away from something you could reasonably consider solely memorization - there's a separate issue of reproduction though.
"novel" here depends on what you mean. Could an LLM produce output that is unique that both it and no one else has seen before, possibly yes. Could that output have perceived or emotional value to people, sure. Related challenge: Is a random encryption key generated by a csprng novel?
In the case of the US copyright office, if there wasn't sufficient human involvement in the production then the output is not copyrightable and how "novel" it is does not matter - but that doesn't necessarily impact a prior production by a human that is (whether a copy or not). Novel also only matters in a subset of the many fractured areas of copyright laws affecting the space of this form of digital replication. The copyright office wrote: https://www.copyright.gov/ai/Copyright-and-Artificial-Intell....
Where I imagine this approximately ends up is some set of tests that are oriented around how relevant to the whole the "copy" is, that is, it may not matter whether the method of production involved "copying", but may more matter if the whole works in which it is included are at large a copy, or, if the area contested as a copy, if it could be replaced with something novel, and it is a small enough piece of the whole, then it may not be able to meet some bar of material value to the whole to be relevant - that there is no harmful infringement, or similarly could cross into some notion of fair use.
I don't see much sanity in a world where small snippets become an issue.
I think if models were regularly producing thousands of tokens of exactly duplicate content that's probably an issue.
I've not seen evidence of the latter outside of research that very deliberately performs active search for high probability cases (such as building suffix tree indices over training sets then searching for outputs based on guidance from the index). That's very different from arbitrary work prompts doing the same, and the models have various defensive trainings and wrappings attempting to further minimize reproductive behavior. On the one hand you have research metrics like 3.6 bits per parameter of recoverable input, on the other hand that represents a very small slice of the training set, and many such reproductions requiring strongly crafted and long prompts - meaning that for arbitrary real world interaction the chance of large scale overlap is small.
By novel, I mean if I ask a model to write some lyrics or code and it produces pre-existing code or lyrics, is it novel and legally safe to use because the pre-existing code or lyrics aren’t precisely encoded in a large enough model, and therefore legally not a reproduction just coincidentally identical.
No. I don't think "novelty" would be relevant in such a case. How much risk you have depends on many factors, including what you mean by "use". If you mean sell, and you're successful, you're at risk. That would be true even if it's not the same as other content but just similar. Copyright provides little to no protection from legal costs if someone is motivated to bring a case at you.
In the West you are free to make something that everyone thinks is a “derivative piece of trash” and still call it yours; and sometimes it will turn out to be a hit because, well, it turns out that in real life no one can reliably tell what is and what isn’t trash[0]—if it was possible, art as we know it would not exist. Sometimes what is trash to you is a cult experimental track to me, because people are different.
On that note, I am not sure why creators in so many industries are sitting around while they are being more or less ripped off by massive corporations, when music has got it right.
— Do you want to make a cover song? Go ahead. You can even copyright it! The original composer still gets paid.
— Do you want to make a transformative derivative work (change the composition, really alter the style, edit the lyrics)? Go ahead, just damn better make sure you license it first. …and you can copyright your derivative work, too. …and the original composer still gets credit in your copyright.
The current wave of LLM-induced AI hype really made the tech crowd bend itself in knots trying to paint this as an unsolvable problem that requires IP abuse, or not a problem because it’s all mostly “derivative bits of trash” (at least the bits they don’t like, anyway), argue in courts how it’s transformative, etc., while the most straightforward solution keeps staring them in the face. The only problem is that this solution does not scale, and if there’s anything the industry in which “Do Things That Don’t Scale” is the title of a hit essay hates then that would be doing things that don’t scale.
[0] It should be clarified that if art is considered (as I do) fundamentally a mechanism of self-expression then there is, of course, no trash and the whole point is moot.
There's an whole genre of musicians focusing only on creating royalty free covers of popular songs so the music can be used in suggestive ways while avoiding royalties.
> There's an whole genre of musicians focusing only on creating royalty free covers
There is no such thing as a “royalty free cover”. Either it is a full on faithful cover, which you can perform as long as license fees are paid, and in which case both the performer and the original songwriter get royalties, or it is a “transformative cover” which requires negotiation with the publisher/rights owner (and in that case IP ownership will probably be split between songwriter and performer depending on their agreement).
(Not an IP lawyer myself so someone can correct me.)
Furthermore, in countries where I know how it works as a venue owner you pay the rights organization a fixed sum per month or year and you are good to go and play any track you want. It thus makes no difference to you whether you play the original or a cover.
Have you considered that it is simply singers-performers who like to sing and would like to earn a bit of money from it, but don’t have many original songs if their own?
> It's parasitism of art
If we assume covers are parasitism of art, by that logic would your comment, which is very similar to dozens I have seen on this topic in recent months, be parasitism of discourse?
Jokes aside, a significant number of covers I have heard at cafes over years are actually quite decent, and I would certainly not call that parasitic in any way.
Even pretending they were, if you compare between artists specialising in covers and big tech trying to expropriate IP, insert itself as a middleman and arbiter for information access, devalue art for profit, etc., I am not sure they are even close in terms of the scale of parasitism.
> Have you considered that it is simply singers-performers who like to sing and would like to earn a bit of money from it, but don’t have many original songs if their own?
Or, maybe you start to pay attention?
They are selling their songs cheaper for TV, radio or ads.
> Even pretending they were, if you compare between artists specialising in covers and big tech trying to expropriate IP
> They are selling their songs cheaper for TV, radio or ads.
I guess that somehow refutes the points I made, I just can’t see how.
Radio stations, like the aforementioned venue owners, pay the rights organizations a flat annual fee. TV programs do need to license these songs (as unlike simple cover here the use is substantially transformative), but again: 1) it does not rip off songwriters (holder of songwriter rights for a song gets royalties for performance of its covers, songwriter has a say in any such licensing agreement), and 2) often a cover is a specifically considered and selected choice: it can be miles better fitting for a scene than the original (just remember Motion Picture Soundtrack in that Westworld scene), and unlike the original it does not tend to make the scene all about itself so much. It feels like you are yet to demonstrate how it is particularly parasitic.
Edit: I mean honest covers; modifying a song a little bit and passing it as original should be very sueable by the rights holder and I would be very surprised if Spotify decided to do that even if they fired their entire legal department and replaced it with one LLM chatbot.
I really doubt you would ever license any specific songs as a cafe business. You should be able to pay a fixed fee to a PRO and have a blanket license to play almost anything. Is it so expensive in the US, or perhaps they do not know that this is an option? If the former, and those cover artists help those bars keep their expenses low and offer you better experience while charging less—working with the system, without ripping off the original artists who still get paid their royalty—does it seem particularly parasitic?
A restaurant / cafe may pay a fixed fee and get access to a specific catalog of songs (performances). The fee depends on what the catalog contains. As you can imagine, paying for the right to only play instrumental versions of songs (no singers, no lyrics) is significantly cheaper. Or, having performances of songs by unknown people.
Two countries where I know how it works from a venue business owner perspective work this way. The fees seemed pretty mild, that’s why I asked if it’s too expensive in your country (which I guess is not US).
These can all be owned by different people or the same person. The "royalty free covers" you mention are people abusing the rights of one of those. They're not avoiding royalties, they just havn't been caught yet.
I believe performance of a cover still results in relevant royalties paid to the original songwriter, just sans the performance fee, which does not strike me as a terrible ripoff (after all, a cover did take effort to arrange and perform).
What this person is talking about is they write “tvinkle tvinkle ittle stawr” instead of the real lyrics (basically just writing the words phonetically and/or misspelled) to try and bypass the law through “technicalities” that wouldn’t stand up in court.
I doubt so for a few reasons based on how they described this alleged parasitic activity, but mainly because the commenter alluded to Spotify doing this. Would be very surprising if they decided to do something so blatantly illegal, when they could keep extracting money by the truckload with their regular shady shenanigans that do not cross that legality line so obviously.
Regarding what you described, I don’t think I encountered this in the wild enough to remember. IANAL but if not cleared/registered properly as a cover it doesn’t seem to be a workaround or abuse, but would probably be found straight up illegal if the rights holder or relevant rights organization cares to sue. In this case, all I can say is “yes, some people do illegal stuff”. The system largely works.
> For a large LLM I think the science in the end will demonstrate that verbatim reproduction is not coming from verbatim recording
We don't need all this (seemingly pretty good) analysis. We already know what everyone thinks: no relevant AI company has had their codebase or other IP scraped by AI bots they don't control, and there's no way they'd allow that to happen, because they don't want an AI bot they don't control to reproduce their IP without constraint. But they'll turn right around and be like, "for the sake of the future, we have to ingest all data... except no one can ingest our data, of course". :rolleyes:
There are only so many ways to code quite a few things. My classmate and I once got in trouble in high school for having identical code for one of the tasks at a coding competition, down to variable names and indentation. There is no way he could or would steal my code, and I sure didn't steal his.
> In order to keep SQLite completely free and unencumbered by copyright, the project does not accept patches. If you would like to suggest a change and you include a patch as a proof-of-concept, that would be great. However, please do not be offended if we rewrite your patch from scratch.
Or you know, they just feel like code should be free. Like beer should be free.
We didn't have this whole issue 20 years ago because nobody gave a shit. If your code was public, and on the internet, it was free for everyone to use by definition.
An LLM can be used for a clean room design so long as all (ALL) of its training data is in the clean room (and consequently does not contain the copyrighted work being reverse engineered).
An LLM trained on the Internet-at-large is also presumably suitable for a clean room design if it can be shown that its training completed prior to the existence of the work being duplicated, and thus could not have been contaminated.
This doesn't detract from the core of your point, that LLM output may be copyright-contaminated by LLM training data. Yes, but that doesn't necessarily mean that an LLM output cannot be a valid clean-room reverse engineer.
> An LLM trained on the Internet-at-large is also presumably suitable for a clean room design if it can be shown that its training completed prior to the existence of the work being duplicated, and thus could not have been contaminated.
This is assuming that you are only concerned with a particular work when you need to be sure that you are not copying any work that might be copyrighted without making sure to have a valid license that you are abiding by.
The "clean room" in "clean room reverse engineering" refers to a particular set of trade secrets, yes. You could have a clean room and still infringe if an employee in the room copied any work they had ever seen.
The clean room has to do with licenses and trade secrets, not copyright.
There is also IP taint when using "AI". We're just pretending that there's not.
If someone came to you and said "good news: I memorized the code of all the open source projects in this space, and can regurgitate it on command", you would be smart to ban them from working on code at your company.
But with "AI", we make up a bunch of rationalizations. ("I'm doing AI agentic generative AI workflow boilerplate 10x gettin it done AI did I say AI yet!")
And we pretend the person never said that they're just loosely laundering GPL and other code in a way that rightly would be existentially toxic to an IP-based company.
Courts (at least in the US) have already ruled that use of ingested data for training is transformative. There’s lots of details to figure, but the genie is out of the bottle.
Sure it’s a big hill to climb in rethinking IP laws to align with a societal desire that generating IP continue to be a viable economic work product, but that is what’s necessary.
> Courts (at least in the US) have already ruled that use of ingested data for training is transformative
Yes, the training of the model itself is (or should be) a transformative act so you can train a model on whatever you have legal access to view.
However, that doesn't mean that the output of the model is automatically not infringing. If the model is prompted to create a copy of some copyrighted work, that is (or should be) still a violation.
Just like memorizing a book isn't infringment but reproducing a book from memory is.
The fact that GitHub’s Copilot has an enterprise feature that matches model output against code having certain licenses - in order to prevent you from using it, with a notification - suggests the model outputs are at least potentially infringing.
If MS were compelled to reveal how these completions are generated, there’s at least a possibility that they directly use public repositories to source text chunks that their “model” suggested were relevant (quoted as it could be more than just a model, like vector or search databases or some other orchestration across multiple workloads).
> suggests the model outputs are at least potentially infringing.
The only thing it suggests is that they recognize that a subset of users worry about it. Whether or not GitHub worries about it any further isn’t suggested.
Don’t think about it from an actual “rights” perspective. Think about the entire copyright issue as a “too big to fail” issue.
Not really, no. If you’re specifically referring to, say, GPL or BSD or other Open Source licenses, it’s a bit more unsettled, but software licensing as a whole has several decades of case law at this point.
No more so than regurgitating an entire book. While it could technically be possible in the case of certain repos that are ubiquitous on the internet (and therefore overrepresented in training data to the point that they are "regurgitated" verbatim, in whole), it is extremely unlikely and would only occur after deliberate prompting. The NYT suit against Open AI shows (in discovery) that the NYT was only able to get partial results after deliberately prompting the model with portions of the text they were trying to force it to regurgitate.
So. Yes, technically possible. But impossible by accident. Furthermore when you make this argument you reveal that you don't understand how these models work. They do not simply compress all the data they were trained on into a tiny storable version. They are effectively multiplication matrices that allow math to be done to predict the most likely next token (read: 2-3 Unicode characters) given some input.
So the model does not "contain" code. It "contains" a way of doing calculations for predicting what text comes next.
Finally, let's say that it is possible that the model does spit out not entire works, but a handful of lines of code that appear in some codebase.
This does not constitute copyright infringement, as the lines in question a) represent a tiny portion of the whole work (and copyright only protecst against the reduplication of whole works or siginficant portions of the work), and B) there are a limited number of ways to accomplish a certain function and it is not only possible but inevitable that two devs working independently could arrive at the same implementation. Therefore using an identical implementation (which is what this case would be) of a part of a work is no more illegal than the use of a certain chord progression or melodic phrasing or drum rhythm. Courts have ruled about this thoroughly.
Yes, that is one of those works that is over-represented in the training data, as I explained in the part of the comment you clearly did not comprehend.
> Sure it’s a big hill to climb in rethinking IP laws to align with a societal desire that generating IP continue to be a viable economic work product, but that is what’s necessary.
Well, AI can perhaps solve the problem it created here: generated IP with AI is much cheaper than with humans, so it will be viable even at lower payoffs.
Less cynical: you can use trade secrets to protect your IP. You can host your software and only let customers interact with it remotely, like what Google (mostly) does.
Of course, this is a very software-centric view. You can't 'protect' eg books or music in this way.
Eh, they figured out how to copyright photographs, where the human only provides a few bits (setting up the scene, deciding when to pull the trigger etc); so stretching a few bits of human input to cover the whole output of an AI should also be doable with sufficiently well paid lawyers.
Tell that to Reddit. They’re AI translating user posts and serving it up as separate Google search results. I don’t remember if Reddit claims copyright on user-submitted content, or on its AI translations, but I don’t think Reddit is paying ad share like X is, either, so it kind of doesn’t matter to the user, as they’re (still) not getting paid, even as Reddit collects money for every ad shown/clicked. Even if OP did write it, an AI translated the version shown.
reddit is a user hostile company, have been forever, always will be. they take rights over your content, farm things about you, sell data, do invasive things in the mobile apps, use creepware cookies, etc.
Excerpt from the user agreement:
When Your Content is created with or submitted to the Services, you grant us a worldwide, royalty-free, perpetual, irrevocable, non-exclusive, transferable, and sublicensable license to use, copy, modify, adapt, prepare derivative works of, distribute, store, perform, and display Your Content and any name, username, voice, or likeness provided in connection with Your Content in all media formats and channels now known or later developed anywhere in the world. This license includes the right for us to make Your Content available for syndication, broadcast, distribution, or publication by other companies, organizations, or individuals who partner with Reddit. For example, this license includes the right to use Your Content to train AI and machine learning models, as further described in our Public Content Policy. You also agree that we may remove metadata associated with Your Content, and you irrevocably waive any claims and assertions of moral rights or attribution with respect to Your Content.
People put their heads in the sand over reddit for some reason, but it's worse than FAANG.
With respect to the content or other materials you upload through the Site or share with other users or recipients (collectively, “User Content”), you represent and warrant that you own all right, title and interest in and to such User Content, including, without limitation, all copyrights and rights of publicity contained therein. With respect to the content or other materials you upload through the Site or share with other users or recipients (collectively, “User Content”), you represent and warrant that you own all right, title and interest in and to such User Content, including, without limitation, all copyrights and rights of publicity contained therein. By uploading any User Content you hereby grant and will grant Y Combinator and its affiliated companies a nonexclusive, worldwide, royalty free, fully paid up, transferable, sublicensable, perpetual, irrevocable license to copy, display, upload, perform, distribute, store, modify and otherwise use your User Content for any Y Combinator-related purpose in any form, medium or technology now known or later developed.
To a certain reading, this is user-centric: it’s increasing the size of the audience pool beyond that of shared language speakers and readers to the entire literate human race. This is an important point to acknowledge, because every silver lining has its cloud.
It's not about being mandatory. It's about having a privileged position and using it to extract value from people.
The nature of network effects is such that once a site gets as big as reddit (or facebook or tiktok or whichever), it's nearly impossible for competition to take over in the same design space.
Many communities (both small and large) are only present on specific platforms (sometimes only one) and if you want to participate you have to accept their terms or exclude yourself socially.
If they have a privileged position, it is earned and freely given. No one is obligated to use the site. The issue is more one of the commons being enclosed and encircled by corporate interests, and then branded and commodified. Once the deed is done, there is no reason for folks to leave, because everyone they know is there.
Most communities on Reddit that I’d care to be a part of have additional places to gather, but I do take your point that there are few good alternatives to r/jailbreak, for example.
The host always sets its own rules. How else could anything actually get done? The coordination problem is hard enough as it is. It’s a wonder that social media exists at all.
Gatekeepers will always exist adjacent to the point of entry, otherwise every site turns extremist and becomes overrun with scammers and spammers.
I don't see why verbatim or not should matter at all.
How complex does a mechanical transformation have to be to not be considered plagiarism, copyright infringement or parasitism?
If somebody writes a GPL-licensed program, is it enough to change all variable and function names to get rid of those pesky users' rights? Do you have to change the order of functions? Do you have to convert it to a different language? Surely nobody would claim c2rust is transformative even though the resulting code can be wildly different if you apply enough mechanical transformations.
All LLMs do is make the mechanical transformations 1) probabilistic 2) opaque 3) all at once 4) using multiple projects as a source.
> How complex does a mechanical transformation have to be to not be considered plagiarism, copyright infringement or parasitism?
Legally speaking, this depends from domain to domain. But consider for example extracting facts from several biology textbooks, and then delivering those facts to the user in the characteristic ChatGPT tone that is distinguishable from the style of each source textbook. You can then be quite assured that courts will not find that you have infringed on copyright.
> Courts (at least in the US) have already ruled that use of ingested data for training is transformative.
If you have code that happens to be identical to some else's code or implements someone's proprietary algorithm, you're going to lose in court even if you claim an "AI" gave it to you.
AI is training on private Github repos and coughing them up. I've had it regurgitate a very well written piece of code to do a particular computational geometry algorithm. It presented perfect, idiomatic Python with perfect tests that caught all the degenerate cases. That was obviously proprietary code--no amount of searching came up with anything even remotely close (it's why I asked the AI, after all).
>If you have code that happens to be identical to some else's code or implements someone's proprietary algorithm, you're going to lose in court even if you claim an "AI" gave it to you.
Not for a dozen lines here or there, even if it could be found and identified in a massive code base. That’s like quoting a paragraph of a book in another book, non infringing.
For the second half of your comment it sounds like you’re saying you got results that were too good to be AI- that’s a bit “no true Scotsman”, at least without more detail. But implementing an algorithm, even a complex one, is very much something an LLM can do. Algorithms are much better defined and scoped natural language, and LLMs do a reasonable job of translating to languages. An algorithm is a narrow subset of that task type with better defined context and syntax.
> Not for a dozen lines here or there, even if it could be found and identified in a massive code base. That’s like quoting a paragraph of a book in another book, non infringing.
It's potentially non-infringing in a book if you quote it in a plausible way, and properly.
If you copy&paste a paragraph from another book into yours, it's infringing, and a career-ending scandal. There's plenty of precedent on that.
Just like if you manually copied a function out of some GPL code and pasted it into your own.
What will happen when company A implements algorithm X based on AI output, company B does the same and company A claims that it is proprietary code and takes company B to court?
It cannot do anything on its own, it's just a (very complex, probabilistic) mechanical transformation (including interpolation) of training data and a prompt.
Advertising autocomplete as AI was a genius move because people start humanizing it and look for human-centric patterns.
Thinking A"I" can do anything on its own is like seeing faces in rocks on Mars.
The idea that something that can't handle simple algorithms (e.g. counting the number of times a letter occurs in a word) could magically churn out far more advanced algorithms complete with tests is… well it's a bit of a stretch.
LLMs aren't good at rote memorization. They can't even get quotations of humans right.
It's easier for the LLM to rewrite an idiomatic computational geometry algorithm from scratch in a language it understands well like Python. Entire computational geometry textbooks and research papers are in its knowledge base. It doesn't have to copy some proprietary implementation.
That seems a real stretch. GPT 5 just invented new math for reference. What you are saying would be equivalent to saying that this math was obviously in some paper that mathematician did not know about. Maybe true, but it's a far reach.
It invented "new math" as much as I invented "new food" when I was cooking yesterday. It did a series of quite complicated calculations that would take a well trained human several hours or even days to do - still impressive, but no it's not new maths.
Obviously not ChatGPT. But ChatGPT isn't the sharpest stick on the block by a significant margin. It is a mistake to judge what AIs can do based on what ChatGPT does.
This would be the first time ever that an LLM has discovered new knowledge, but the far reach is that the information does appear in the training data?
They've been doing it for a while. Gemini has also discovered new math and new algorithms.
There is an entire research field of scientific discovery using LLMs together with sub-disciplines for the various specialization. LLMs routinely discover new things.
I hadn't heard of that, so I did some searching and the single source for the claim I can find is a Google white paper. That doesn't automatically mean it's false, of course, but it is curious that the only people ostensibly showing LLMs discover new things are the companies offering the LLMs.
Citation needed, and I call bullshit. Unless you mean that they hallucinate useless algorithms that do not work, which they do.
LLMs do not have an internal model for manipulating mathematical objects. They cannot, by design, come up with new algorithms unless they are very nearly the same as some other algorithm. I'm a computer science researcher and have not heard of a single algorithm created by LLM.
This article is about the same thing I mentioned in a sibling comment. I personally don't find an unreplicated Google white paper to be compelling evidence.
The AI coming up with it? When Google claimed their Wizard of Oz show at the Las Vegas Sphere was AI-generated, a ton of VFX artists spoke up to say they'd spent months of human labor working on it. Forgive me for not giving the benefit of the doubt to a company that has a vested interest in making their AI seem more powerful, and a track record of lying to do so.
Maybe I should clarify: Society, in general, supports the idea that writers, artists, film makers, coders, etc— everyone who creates IP- should have a place in the economy. Basically just that it should be possible to make a living and have a career at it. It can be spun different ways, and those differences are important, but this is the basic thing.
This doesn’t seem like a disputable statement to me. For anyone who thinks actors’ likenesses, authors’ words, all of it- that all and everything should be up for grabs once written or put anywhere in public, that is not a widely held opinion.
Once that’s established, it all comes down to implementation details.
If you mean the ruling has absolutely no applicability when it comes to using the model then, no, that is incorrect:
Judge Alsup, in his ruling, specifically likened the process to reading text and then using the knowledge to write something else. That’s training and use.
A compression algorithm doesn't transform the data it stores it in a different format. Storing a story in a txt file vs word file doesn't transform the data.
An llm is looking at the shape of words and ideas over scale and using that to provide answers.
No a compression algorithm does transform the data, particularly lossy ones. The pixels stored in the output are not in the input, they're new pixels. That's why you can't uncompress a jpeg. Its a new image that just happens to look like the original. But it even might not - some jpegs are so deep fried they become their own form of art. This is very popular in meme culture.
The only difference, really, is we know how a JPEG algorithm works. If I wanted to, I could painstakingly make a jpeg by hand. We don't know how LLMs work.
"see" and "copy" are two different things. It's fine to look at StackOverflow to understand the solution to a problem. It's not fine to copy and paste from StackOverflow and ignore its license or attribution.
Content on StackOverflow is under CC-by-sa, version depends on the date it was submitted: https://stackoverflow.com/help/licensing . (It's really unfortunate that they didn't pick license compatible with code; at one point they started to move to the MIT license for code, but then didn't follow through on it.)
That's a fair distinction. For the specific case of CC-by-sa 4.0, it's possible to convert to GPLv3. That doesn't help with prior versions of CC-by-sa.
So, for the specific case of material contributed to StackOverflow on or after 2018-05-02, it's possible to use it under GPLv3 (including appropriate attribution), so any project compatible with GPLv3 can copy it with attribution. Any material before that point is not safe to copy.
It does help with prior versions, since you can use CC BY-SA 3.0 material under CC BY-SA 4.0, which is GPLv3-compatible. (See https://meta.stackexchange.com/a/337742/308065.) It doesn't necessarily help with future versions.
I stand corrected, thank you! I forgot that CC-by-sa included an automatic upgrade clause, unlike many other licenses that require an explicit "or any later version" to do that.
> The reality is that programmers are going to see other programmers code.
You're certainly correct. It's also true that companies are going to sue over it. There's no reason to make yourself an easy lawsuit target, if it's trivial to avoid it.
There's a reason why clean-room reverse engineering exists and it's to ensure that you can't commit copyright infringement by knowing about the code you are trying to reimplement.
That's not really how LLMs are used, unless we're planning on classing gitignores as IP
LLMs are interesting because they can combine things they learn from multiple projects into a new language that doesn't feature in any of them, and pick up details from your request.
Unless you're schizophrenic enough to insist that you never even see other code it's just not a realistic problem
Honestly I've had big arguments about this IP stuff before and unless you actually have a lawyer specifically go after something or very obviously violate the GPL it's just a tactic for people to slow people they don't like down. People find a way to invent HR departments fractally.
> If someone came to you and said "good news: I memorized the code of all the open source projects in this space, and can regurgitate it on command", you would be smart to ban them from working on code at your company.
If you find a human that did that send them my way, I'll hire them.
> There is also IP taint when using "AI". We're just pretending that there's not.
I don't think anyone who's not monetarily incentivize to pretend there are IP/Copyright issues actually thinks there are. Luckily everyone is for the most part just ignoring them and the legal system is working well and not allowing them an inch to stop progress.
> I don't think anyone who's not monetarily incentivize to pretend there are IP/Copyright issues actually thinks there are.
Why do you think that about people who disagree with you? You're responding directly to someone who's said they think there's issues, and not pretending. Do you think they're lying? Did you not read what they said?
And AFAICT a lot of other people think similarly to me.
The perverse incentives to rationalize are on the side of the people looking to exploit the confusion, not the people who are saying "wait a minute, what you're actually doing is..."
So a gold rush person claiming opponents must be pretending because of incentives... seems like the category of "every accusation is a confession".
Putting aside the notion that a monetary incentive is invalid for some reason, there are also plenty of open source authors who are pretty unimpressed that they only ask for attribution and even that's being laundered away via weights. If their licence isn't being respected that's certainly a legal issue.
Learning isn't stealing, but learning can absolutely lead to stealing. (Edit: This is why bring able to demostrate that you did not learn from a copyrighted work when trying to replicate and compete with it can be an important defense.)
The amount of IP risk caused by USING (not training) AI models to produce code, especially wholesale commercial code that competes with code that was contained in the training data, is poorly understood.
> I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of
I really appreciate this point from mitchellh. Giving thoughtful constructive feedback to help a junior developer improve is a gift. Yet it would be a waste of time if the PR submitter is just going to pass it to an AI without learning from it.
> Junior developers are entering a workforce where they will never not be using AI
This remark seems very US-centric to me. In my observation, many people are much more skeptical concerning whether AI is actually useful beyond some gimmicky applications.
> If you are using *any kind of AI assistance* to contribute to Ghostty, it must be disclosed in the pull request.
This is sufficiently confusing that someone is asking if this applies to tab completion. They commit actually says
> trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases.
So if you take this literally you're going to be disclosing every yasnippet expansion that completes boilerplate.
The policy as written isn't sensible and I don't think it's entirely coming from a sensible place.
Junior developers need to learn how to code with AI because that's what coding is now. Not that he has to help them. But it does read a bit weird to toot your horn about how important it is to be helpful until it comes to helping people understand how to navigate the current environment then it's not worth your time.
The question is going to be: is it a USEFUL signal. I suspect not. And frankly, to be honest, as a senior developer who uses AI assistants routinely, I would consider it a serious disincentive to actually submit a PR in the first place to a repository that has such a policy. Submitting a good PR is hard work. Often an order of magnitude more work than the fix itself. If I think that a repository is going to not accept my PR just because I use a coding assistant, I'm going to be much less inclined to submit a PR at all.
I’m loving today. HN’s front page is filled with some good sources today. No nonsense sensationalism or preaching AI doom, but more realistic experiences.
I’ve completely turned off AI assist on my personal computer and only use AI assist sparingly on my work computer. It is so bad at compound work. AI assist is great at atomic work. The rest should be handled by humans and use AI wisely. It all boils down back to human intelligence. AI is only as smart as the human handling it. That’s the bottom line.
I think I'm slowly coming around to this viewpoint too. I really just couldn't understand how so many people were having widely different experiences. AI isn't magic; how could I have expected all the people I've worked with who struggle to explain stuff to team members, who have near perfect context, to manage to get anything valuable across to an AI?
I was original pretty optimistic that AI would allow most engineers to operate at a higher level but it really seems like instead it's going to massively exacerbate the difference between an ok engineer and a great engineer. Not really sure how I feel about that yet but at-least I understand now why some people think the stuff is useless.
One of my mental models is that the notion of "effective engineer" used to mean "effective software developer" whether or not they were good at system design.
Now, an "effective engineer" can be a less battle-tested software developer, but they must be good at system design.
(And by system design, I don't just mean architecture diagrams: it's a personal culture of constantly questioning and innovating around "let's think critically to see what might go wrong when all these assumptions collide, and if one of them ends up being incorrect." Because AI will only suggest those things for cut-and-dry situations where a bug is apparent from a few files' context, and no ambitious idea is fully that cut-and-dry.)
The set of effective engineers is thus shifting - and it's not at all a valid assumption that every formerly good developer will see their productivity skyrocket.
Great Engineer + AI = Great Engineer++ (Where a great engineer isn't just someone who is a great coder, they also are a great communicator & collaborator, and love to learn)
I recently watched a mid-level engineer use AI to summarize some our code, and he had it put together a big document describing all the various methods in a file, what they're used for, and so forth. It looked to me like a huge waste of time, as the code itself was already very readable (I say this as someone who recently joined the project), and the "documentation" the AI spit out wasn't that different than what you'd get just by running pydoc.
He took a couple days doing this, which was shocking to me. Such a waste of time that would have been better spent reading the code and improving any missing documentation - and most importantly asking teammates about necessary context that couldn't just be inferred from the code.
I hate to break it to you, but this guy probably wasn’t working at all. That sounds like a pretense to goof off.
Now I could believe an intern would do such a thing. I’ve seen a structural engineer intern spend four weeks creating a finite element model of a single concrete vault. he could have treated the top deck as a concrete beam used conservative assumptions about the loading and solved it with pen and paper in 30 minutes.
The same people who just copy-pasted stack overflow answers and didn't understand why or how things work are now using AI to create stuff that they also don't understand.
lol. I am SO glad I don't have to go to StackExchange anymore. There is something toxically awful about using advice from a thread that starts with "Why doesn't my code work?".
Another common synonym for mediocre: has no place on a software development team. Not technically correct, admittedly, but that's how I read that word in an software engineering context. Adequate is not good enough.
“Mediocre” is one of those words where common parlance doesn’t quite line up with the textbook definition. e.g. from the Oxford English Dictionary: “Of middling quality; neither bad nor good...”
This fits my observations as well. With the exception that it’s sometimes the really sharp engineers who can do wonders themselves who aren’t really great at communication. AI really needs you to be verbose, and a lot of people just can’t.
At the moment, AI tools are particularly useful for people who feel comfortable browsing through large amounts of text, intuitively nudging the machine this way and that until arriving at a valuable outcome.
However, that way of working can be exasperating for those who prefer a more deterministic approach, and who may feel frustrated by the sheer amount of slightly incorrect stuff being generated by the machine.
I've been struggling to apply AI on any large scale at work. I was beginning to wonder if it was me.
But then my wife sort of handed me a project that previously I would have just said no to, a particular Android app for the family. I have instances of all the various Android technologies under my belt, that is, I've used GUI toolkits, I've used general purpose programming languages, I've used databases, etc, but with the possible exception of SQLite (which even that is accessed through an ORM), I don't know any of the specific technologies involved with Android now. I have never used Kotlin; I've got enough experience that I can pretty much piece it together when I'm reading it but I can't write it. Never used the Android UI toolkit, services, permissions, media APIs, ORMs, build system, etc.
I know from many previous experiences that A: I could definitely learn how to do this but B: it would be a many-week project and in the end I wouldn't really be able to leverage any of the Android knowledge I would get for much else.
So I figured this was a good chance to take this stuff for a spin in a really hard way.
I'm about eight hours in and nearly done enough for the family; I need about another 2 hours to hit that mark, maybe 4 to really polish it. Probably another 8-12 hours and I'd have it brushed up to a rough commercial product level for a simple, single-purpose app. It's really impressive.
And I'm now convinced it's not just that I'm too old a fogey to pick it up, which is, you know, a bit of a relief.
It's just that it works really well in some domains, and not so much in others. My current work project is working through decades of organically-grown cruft owned by 5 different teams, most of which don't even have a person on them that understands the cruft in question, and trying to pull it all together into one system where it belongs. I've been able to use AI here and there for some stuff that is still pretty impressive, like translating some stuff into psuedocode for my reference, and AI-powered autocomplete is definitely impressive when it correctly guesses the next 10 lines I was going to type effectively letter-for-letter. But I haven't gotten that large-scale win where I just type a tiny prompt in and see the outsized results from it.
I think that's because I'm working in a domain where the code I'm writing is already roughly the size of the prompt I'd have to give, at least in terms of the "payload" of the work I'm trying to do, because of the level of detail and maturity of the code base. There's no single sentence I can type that an AI can essentially decompress into 250 lines of code, pulling in the correct 4 new libraries, and adding it all to the build system the way that Gemini in Android Studio could decompress "I would like to store user settings with a UI to set the user's name, and then display it on the home page".
I think I recommend this approach to anyone who wants to give this approach a fair shake - try it in a language and environment you know nothing about and so aren't tempted to keep taking the wheel. The AI is almost the only tool I have in that environment, certainly the only one for writing code, so I'm forced to really exercise the AI.
> try it in a language and environment you know nothing about and so aren't tempted to keep taking the wheel.
That's a good insights. Its almost like to use AI tools effectively, one needs to stop caring about the little things you'd get caught up in if you were already familiar and proficient in a stack. Style guidelines, a certain idiomatic way to do things, naming conventions, etc.
A lot like how I've stopped organizing digital files into folders, sub folders etc (along with other content) and now I just just rely on search. Everything is a flat structure, I don't care where its stored or how it's organized as long as I can just search for it, that's what the computer is for, to keep track for me so I don't have to waste time organizing it myself.
Like wise for the code Generative AI produces. I don't need to care about the code itself. As long as its correct, not insecure, and performant, it's fine.
It's not 100% there yet, I still do have to go in and touch the code, but ideally I shouldn't have to, nor should I have to care what the actual code looks like, just the result of it. Let the computer manage that, not me. My role should be the system design and specification, not writing the code.
> Its almost like to use AI tools effectively, one needs to stop caring about the little things you'd get caught up in if you were already familiar and proficient in a stack. Style guidelines, a certain idiomatic way to do things, naming conventions, etc.
This makes me a little sad. Part of the joy of writing software is expressing yourself through caring about these little things. Stylistic coherence, adhering to consistent naming conventions, aligning code blocks, consistently applying patterns, respecting the way the language and platform's APIs work together rather than fighting it... heck, even tiny things like alphabetizing header declarations. None of these things make the finished product better/faster/more reliable, but all of these demonstrate something about the author: What he believes in. That he is willing to sand, polish and beautify the back of the cabinet that nobody is going to see. As Steve Jobs said:
"Even though it faces the wall and nobody will ever see it. You’ll know it’s there, so you’re going to use a beautiful piece of wood on the back. For you to sleep well at night, the aesthetic, the quality, has to be carried all the way through."
If you surrender to the AI, you're no longer carrying the aesthetic and quality all the way through. You're abandoning the artistry. You're living with the barf because it works, and because it's much harder to go back and beautify it than it is to build it beautifully from the start.
Not a solved problem yet, admittedly. Performance can be measured externally, but ideally we'll get to a point where certain criteria can be automatically tested/AI output be checked without manual intervention, or even before the AI actually puts text to editor.
What you are describing also seems to align with the idea that greenfield projects are well-suited for AI, whereas brownfield projects are considerably more challenging.
Brownfield projects are more challenging because of all the context and decisions that went into building things that are not directly defined in the code.
I suspect that well-engineered projects with plenty of test coverage and high-quality documentation will be easier to use AI on, just like they're easier for humans to comprehend. But you need to have somebody with the big picture still who can make sure that you don't just turn things into a giant mess once less disciplined people start using AI on a project.
Well said - language (text input) is actually the vehicle you have to transfer neural state to the engine. When you are working in a greenfield project or pure-vibe project, you can get away with most of that neural state being in the "default" probability mode. But in a legacy project, you need significantly more context to contrain the probability distributions a lot closer to the decisions which were made historically otherwise you quickly get into spaghetti-ville as the AI tries to drag the codebase towards its natural ruts.
Also, as soon as the code no longer fits within the context window of an LLM, one must resort to RAG-based solutions, which often leads to a significant decline in quality.
It's a matter of the tools not getting there though. If there was a summarization system that could compress down the structure and history of the system you are working on in a way that could then extract out a half-filled context window of the relevant bits of the code base and architecture for the task (in other words, generate that massive prompt for you), then you might see the same results that you get with Android apps.
The reason being that the boilerplate Android stuff is effectively given for free and not part of the context as it is so heavily represented in the training set, whereas the unique details of your work project is not. But finding a way to provide that context, or better yet fine-tune the model on your codebase, would put you in the same situation and there's no reason for it to not deliver the same results.
That it is not working for you now at your complex work projects is a limitation of tooling, not something fundamental about how AI works.
Aside: Your recommendation is right on. It clicked for me when I took a project that I had spent months of full-time work creating in C++, and rewrote it in idiomatic Go, a language I had never used and knew nothing about. It took only a weekend, and at the end of the project I had reviewed and understood every line of generated code & was now competent enough to write my own simple Go projects without AI help. I went from skeptic to convert right then and there.
I agree that the level of complexity of task it can do is likely to rise over time. I often talk about the "next generation" of AI that will actually be what we were promised LLMs would be, but LLMs architecturally are just not suited for. I think the time is coming when AIs "truly" (for some definition of truly) will understand architecture and systems in a way that LLMs don't and really can't, and will be able to do a lot more things than they can now, though when that will be is hard to guess. Could be next year, or AI could stall out now where it is now for the next 10. Nobody knows.
However, the information-theoretic limitation of expressing what you want and how anyone, AI or otherwise, could turn that into commits, is going to be quite the barrier, because that's fundamental to communication itself. I don't think the skill of "having a very, very precise and detailed understanding of the actual problem" is going anywhere any time soon.
(1) The process of creating "a very, very precise and detailed understanding of the actual problem" is something AI is really good at, when partnered with a human. My use of AI tools got immensely better when I figured out that I should be prompting the AI to turn my vague short request into a detailed prompt, and then I spend a few iteration cycles fixing up before asking the agent to do it.
(2) The other problem of managing context is a search and indexing problem, which we are really, really good at and have lots of tools for, but AI is just so new that these tools haven't been adapted or seen wide use yet. If the limitation of the AI was its internal reasoning or training or something, I would be more skeptical. But the limitation seems to be managing, indexing, compressing, searching, and distilling appropriate context. Which is firmly in the domain of solvable, albeit nontrivial problems.
I don't see the information theoretic barrier you refer to. The amount of information an AI can keep in its context window far exceeds what I have easily accessible to my working memory.
The information theoretic barrier is in the information content of your prompt, not the ability of the AI to expand it.
But then I suppose I should learn from my own experiences and not try to make information theoretic arguments on HN, since it is in that most terrible state where everyone thinks they understand it because they use "bits" all the time, but in fact the average HN denizen knows less than nothing about it because even their definition of "bit" actively misleads them and that's about all they know.
I have a CS theory background. If by prompt you mean the full context provided, then there isn’t an effective limit. Claude now has 1M token context windows. You are not going to fill that with just a task specification. You could easily fill it in a large repo with the accumulated design history and total codebase. However this is also largely static, and could be used for fine-tuning. With fine tuning you’re back to a 1M token task specification for the unique variation of this prompt, and recent changes. You are not going to easily fill that.
I have vibe coded things like a personal finance forecast with interactive graphs in less than 30min of effort. I have 3 kids and would never have considered learning React to do that as a hobby project.
There are so many unusual or one off use cases that would have normally required me to spend several hours locating and reading API documentation that I now just hand off to the AI. I am a big believer in their value. I’m getting more done than ever.
The gist being - language (text input) is actually the vehicle you have to transfer neural state to the engine. When you are working in a greenfield project or pure-vibe project, you can get away with most of that neural state being in the "default" probability mode. But in a legacy project, you need significantly more context to contrain the probability distributions a lot closer to the decisions which were made historically.
Today, I read the following in the concluding sentence of Frederik P. Brooks' essay “No Silver Bullets, Refired"[0]. I am quoting the entire chapter in full because it is so apt and ends with a truly positive message.
> Net on Bullets - Position Unchanged
> So we come back to fundamentals. Complexity is the business we are in, and complexity is what limits us. R. L. Glass, writing in 1988, accurately summarizes my 1995 views:
>> So what, in retrospect, have Parnas and Brooks said to us? That software development is a conceptually tough business. That magic solutions are not just around the corner. That it is time for the practitioner to examine evolutionary improvements rather than to wait—or hope—for revolutionary ones.
>> Some in the software field find this to be a discouraging picture. They are the ones who still thought breakthroughs were near at hand.
>> But some of us—those of us crusty enough to think that we are realists—see this as a breath of fresh air. At last, we can focus on something a little more viable than pie in the sky. Now, perhaps, we can get on with the incremental improvements to software productivity that are possible, rather than waiting for the breakthroughs that are not likely to ever come.[1]
[0]: Brooks, Frederick P.,Jr, The mythical man-month: essays on software engineering (1995), p. 226
[1]: Glass, R. L., "Glass"(column), System Development, (January 1988), pp. 4-5.
Plenty of posts in the style of "I wrote this cool library with AI in a day" were written by really smart devs who are known for shipping good quality library very quickly.
I'm right there with you, and having a similar experience at my day job. We are doing a bit of a "hack week" right now where we allow everyone in the org to experiment in groups with AI tools, especially those that don't regularly use them as part of their work - and we've seen mostly great applications of analytical approaches, guardrails and grounded generation.
It might just be my point of view, but I feel like there's been a sudden paradigm shift back to solid ML from the deluge of chatbot hype nonsense.
Sure. But a smart person using an AI is way smarter than a smart person not using an AI. Also keep in mind that the IQ of various AIs varies dramatically. The Google Search AI, for example, has an IQ in the 80s (and it shows); whereas capable paid AIs consistently score in the 120 IQ range. Not as smart as me, but close enough. And entirely capable of doing in seconds what would take me hours to accomplish, while applying every single one of my 120+ IQ points to the problem at hand. Im my opinion, really smart people delegate.
The way I've been thinking about it is that the human makes the key decisions and then the AI connects the dots.
What's a key decision and what's a dot to connect varies by app and by domain, but the upside is that generally most code by volume is dot connecting (and in some cases it's like 80-90% of the code), so if you draw the lines correctly, huge productivity boosts can be found with little downside.
But if you draw the lines wrong, such that AI is making key decisions, you will have a bad time. In that case, you are usually better off deleting everything it produced and starting again rather than spending time to understand and fix its mistakes.
Things that are typically key decisions:
- database table layout and indexes
- core types
- important dependencies (don't let the AI choose dependencies unless it's low consequence)
- what all the UI screens are and what they contain, user flows, etc.
- color scheme, typography, visual hierarchy
- what to test and not to test (AI will overdo it with unnecessary tests and test complexity if you let it)
- code organization: directory layout, component boundaries, when to DRY
Things that are typically dot connecting:
- database access methods for crud
- API handlers
- client-side code to make API requests
- helpers that restructure data, translate between types, etc.
- deploy scripts/CI and CD
- dev environment setup
- test harness
- test implementation (vs. deciding what to test)
- UI component implementation (once client-side types and data model are in place)
- styling code
- one-off scripts for data cleanup, analytics, etc.
That's not exhaustive on either side, but you get the idea.
AI can be helpful for making the key decisions too, in terms of research, ideation, exploring alternatives, poking holes, etc., but imo the human needs to make the final choices and write the code that corresponds to these decisions either manually or with very close supervision.
I think this seems totally reasonable, the additional context provided is, I think, important to the requirement.
Some of the AI policy statements I have seen come across more as ideology statements. This is much better, saying the reasons for the requirement and offering a path forward. I'd like to see more of this and less "No droids allowed"
It does matter how and where a PR comes from, because reviewers are fallible and finite, so trust enters the equation inevitably. You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.
If trust didn't matter, there wouldn't have been a need for the Linux Kernel team to ban the University of Minnesota for attempting to intentionally smuggle bugs through the PR process as part of an unauthorized social experiment. As it stands, if you / your PRs can't be trusted, they should not even be admitted to the review process.
In an open source project I think you have to start with a baseline assumption of "trust nobody." Exceptions possibly if you know the contributors personally, or have built up trust over years of collaboration.
I wouldn't reject or decline to review a PR just because I don't trust the contributor.
Better to think in terms of distrust rather than trust.
Presumably if a contributor repeatedly made bad PRs that didn't do what they said, introduced bugs, scribbled pointlessly on the codebase, and when you tried to coach or clarify at best they later forgot everything you said and at worst outright gaslit and lied to you about their PRs... you would reject or decline to review their PRs, right? You'd presumably ban the outright.
Well that's exactly what commercial LLM products, with the aid of less sophisticated users, have already done to the maintainers of many large open source projects. It's not that they're not trusted-- they should be distrusted with ample cause.
So what if the above banned contributor kept getting other people to mindlessly submit their work and even proxy communication through -- evading your well earned distrust and bans? Asking people to at least disclose that they were acting on behalf of the distrusted contributor would be the least you would do, I hope? Or even asking them to disclose if and to what extent their work was a collaboration with a distrusted contributor?
The observation that inspired this policy is that if you used AI, it is likely you don't know if the code, the documentation or tests are good or appropriate.
What if you started with good documentation that you personally wrote, you gave that to the agent, and you verified the tests were appropriate and passed?
I'd extrapolate that the OP's view would be: you've still put in less effort, so your PR is less worthy of his attention than someone who'd done the same without using LLMs.
That's a pretty nice offer from one of the most famous and accomplished free software maintainers in the world. He's promising not to take a short-cut reviewing your PR, in exchange for you not taking a short-cut writing it in the first place.
LLMs can't count letters, their writing is boring, and you can trick them into talking gibberish. That is a long way off the Turing test, even if we were fooled for a couple of weeks in 2022.
IMO when people declare that LLMs "pass" at a particular skill, it's a sign that they don't have the taste or experience to judge that skill themselves. Or - when it's CEOs - they have an interest in devaluing it.
So yes if you're trying to fool an experienced open source maintainer with unrefined LLM-generated code, good luck (especially one who's said he doesn't want that).
Would you like to take the Pepsi challenge? Happy to put random code snippets in front of you and see whether you can accurately determine whether it was written by a human or an LLM.
> You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.
No you don’t. You can’t outsource trust determinations. Especially to the people you claim not to trust!
You make the judgement call by looking at the code and your known history of the contributor.
Nobody cares if contributors use an LLM or a magnetic needle to generate code. They care if bad code gets introduced or bad patches waste reviewers’ time.
You’re completely incorrect. People care a lot about where code came from. They need to be able to trust that code you’re contributing was not copied from a project under AGPLv3, if the project you’re contributing to is under a different license.
Stop trying to equate LLM-generated code with indexing-based autocomplete. They’re not the same thing at all: LLM-generated code is equivalent to code copied off Stack Overflow, which is also something you’d better not be attempting to fraudulently pass off as your own work.
I’m not equating any type of code generation. I’m saying that as a maintainer you have to evaluate any submission on the merits, not on a series of yes/no questions provided by the submitter. And your own judgement is influenced by what you know about the submitter.
And I’m saying, as a maintainer, you have to and are doing both, even if you don’t think you are.
For example, you either make your contributors attest that their changes are original or that they have the right to contribute their changes—or you assume this of them and consider it implicit in their submission.
What you (probably) don’t do is welcome contributions that the contributors do not have the right to make.
It doesn’t, it provides an assurance (but not provenance) you didn’t use AI.
Assuring you didn’t include any AGPLv3 code in your contribution is exactly the same kind of assurance. It also doesn’t provide any provenance.
Conflating assurance with provenance is bogus because the former is about making a representation that, if false, exposes the person making it to liability. For most situations that’s sufficient that provenance isn’t needed.
Trust is absolutely a thing. Maintaining an open source project is an unreasonably demanding and thankless job, and it would be even more so if you had to treat every single PR as if it's a high likelihood supply-chain attack.
No, we shouldn't. We live in a society, and that level of distrust is not just unrealistic, it's disastrous. This doesn't mean you should share your house keys with every drive by PR contributor, but neither should you treat every PR as if it's coming from Jia Tan.
> Nobody cares if contributors use an LLM or a magnetic needle to generate code.
That’s exactly opposite of what the author is saying. He mentions that [if the code is not good, or you are a beginner] he will help you get to finish line, but if it’s LLM code, he shouldn’t be putting effort because there’s no human on the other side.
It's not a false equivalence. You can teach a beginner to become an intermediate (and later a master, if they stick to it). You can't teach an LLM to be better. Every piece of feedback you give to an LLM is like screaming into the void - it wastes your time, and doesn't change the LLM one iota.
"Every piece of feedback you give to an LLM is like screaming into the void - it wastes your time, and doesn't change the LLM one iota."
I think you just haven't gotten the hang of it yet, which is fine... the tooling is very immature and hard to get consistent results with. But this isn't a given. Some people do get good, steerable LLM coding setups.
As a maintainer, if you're dealing with a contributor who's sending in AI slop, you have no opportunity to prompt the LLM.
The PR effectively ends up being an extremely high-latency conversation with an LLM, via another human who doesn't have the full context/understanding of the problem.
Steering via prompting isn't the same as fundamentally changing the LLM by teaching, as you can do with humans. I think OP understands this better than you.
Can't tell if you're responding in earnest or not here?
LLMs are trained to be steerable at inference time via context/prompting. Fine tuning is also possible and often used. Both count as "feedback" in my book, and my point is that both can be effective at "changing the LLM" in terms of its behavior at inference time.
And also clearly not what the OP means, who was trying to make a point that tuning the prompt to an otherwise stateless LLM inference job is nothing at all like teaching a human being. Mechanically, computationally, morally or emotionally. For example, humans aren't just tools; giving feedback to LLMs does little to further their agency.
The false equivalence I pointed at earlier was "LLM code => no human on the other side".
The person driving the LLM is a teachable human who can learn what's what's going on and learn to improve the code. It's simply not true that there's no person on the other side of the PR.
The idea that we should be comparing "teaching a human" to "teaching an LLM" is yet another instance of this false equivalence.
It's not inherently pointless to provide feedback on a PR with code written using an LLM, that feedback goes to the person using the LLM tools.
People are swallowing this b.s. marketing mystification of "LLMs as non human entities". But really they're fancy compilers that we have a lot to learn about.
The person operating the LLM is not a meaningfully teachable human when they're not disclosing that they're using an LLM.
IF they disclose what they've done, provided the prompts, etc. then other contributors can help them get better results from the tools. But the feedback is very different than the feedback you'd give a human that actually wrote the code in question, that latter feedback is unlikely to be of much value (and even less likely to persist).
You haven't addressed the primary stated rationale from the linked content: "I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so."
> I wouldn’t really care how someone got to the end result that is a PR.
I can generate 1,000 PRs today against an open source project using AI. I think you do care, you are only thinking about the happy path where someone uses a little AI to draft a well constructed PR.
There's a lot ways AI can be used to quickly overwhelm a project maintainer.
In that case a more correct rule (and probably one that can be automatically enforced) for that issue is a max number of PRs or opened issues per account.
I think this is sane, although possibly not sufficient. Asking people to self-disclose AI usage is not going shield maintainers from a flood of undisclosed AI submissions.
> I can generate 1,000 PRs today against an open source project using AI.
Then perhaps the way you contribute, review, and accept code is fundamentally wrong and needs to change with the times.
It may be that technologies like Github PRs and other VCS patterns are literally obsolete. We've done this before throughout many cycles of technology, and these are the questions we need to ask ourselves as engineers, not stick our heads in the sand and pretend it's 2019.
I don't think throwing out the concept of code reviews and version control is the correct response to a purported rise in low-effort high-volume patches. If anything it's even more required.
If machines can iterate faster than humans, we'll need machines to do the reviewing; that means the testing/QA will be done perhaps by machines which will operate on a spec similar to what Amazon is doing with Kilo.
Before PR's existed we passed around code changes via email. Before containers we installed software on bare metal servers. And before search engines we used message boards. It's not unfathomable that the whole idea of how we contribute and collaborate changes as well. Actually that is likely going to be the /least/ shocking thing in the next few years if acceleration happens (i.e. The entire OS is an LLM that renders pixels, for example)
It's not just about how you got there. At least in the United States according to the Copyright Office... materials produced by artificial intelligence are not eligible for copyright. So, yeah, some people want to know for licensing purposes. I don't think that's the case here, but it is yet another reason to require that kind of disclosure... since if you fail to mention that something was made by AI as part of a compound work you could end up losing copyright over the whole thing. For more details, see [2] (which is part of the larger report on Copyright and AI at [1]).
> • The use of AI tools to assist rather than stand in for human creativity does not affect the availability of copyright protection for the output.
> • Copyright protects the original expression in a work created by a human author, even if the work also includes AI-generated material
> • Human authors are entitled to copyright in their works of authorship that are perceptible in AI-generated outputs, as well as the creative selection, coordination, or arrangement of material in the outputs, or creative modifications of the outputs.
Original expression, yes, however you should've kept reading:
"In the Office’s view, it is well-established that copyright can protect only material that is
the product of human creativity. Most fundamentally, the term “author,” which is used
in both the Constitution and the Copyright Act, excludes non-humans."
"In the case of works containing
AI-generated material, the Office will consider whether the AI contributions are the result
of “mechanical reproduction” or instead of an author’s “own original mental conception,
to which [the author] gave visible form.” 24 The answer will depend on the circumstances,
particularly how the AI tool operates and how it was used to create the final work.25 This
is necessarily a case-by-case inquiry."
"If a work’s traditional elements of authorship were produced by a machine, the work lacks
human authorship and the Office will not register it."
The office has been quite consistent that works containing both human-made and AI-made elements will be registerable only to the extent that they contain human-made elements.
> if you fail to mention that something was made by AI as part of a compound work you could end up losing copyright over the whole thing
The source you linked says the opposite of that: "the inclusion of elements of AI-generated content in a larger human-authored work does not affect the copyrightability of the larger human-authored work as a whole"
The quote you pulled suggests that if the work is majority machine-generated, then it loses copyright protection.
That is, it suggests that even if there are elements of human-generated content in a larger machine-generated work, the combined work as a whole is not eligible for copyright protection. Printed page iii of that PDF talks a bit more about that:
* Copyright does not extend to purely AI-generated material, or material where there is insufficient human control over the expressive elements.
* Whether human contributions to AI-generated outputs are sufficient to constitute authorship must be analyzed on a case-by-case basis.
Just to be sure that I wasn't misremembering, I went through part 2 of the report and back to the original memorandum[1] that was sent out before the full report issued. I've included a few choice quotes to illustrate my point:
"These are no longer hypothetical questions, as the Office is already receiving and
examining applications for registration that claim copyright in AI-generated material. For
example, in 2018 the Office received an application for a visual work that the applicant
described as “autonomously created by a computer algorithm running on a machine.” 7
The application was denied because, based on the applicant’s representations in the
application, the examiner found that the work contained no human authorship. After a
series of administrative appeals, the Office’s Review Board issued a final determination
affirming that the work could not be registered because it was made “without any creative
contribution from a human actor.”"
"More recently, the Office reviewed a registration for a work containing human-authored
elements combined with AI-generated images. In February 2023, the Office concluded
that a graphic novel comprised of human-authored text combined with images generated
by the AI service Midjourney constituted a copyrightable work, but that the individual
images themselves could not be protected by copyright. "
"In the Office’s view, it is well-established that copyright can protect only material that is the product of human creativity. Most fundamentally, the term “author,” which is used in both the Constitution and the Copyright Act, excludes non-humans."
"In the case of works containing AI-generated material, the Office will consider whether the AI contributions are the result of “mechanical reproduction” or instead of an author’s “own original mental conception, to which [the author] gave visible form.” The answer will depend on the circumstances, particularly how the AI tool operates and how it was used to create the final work. This is necessarily a case-by-case inquiry."
"If a work’s traditional elements of authorship were produced by a machine, the work lacks
human authorship and the Office will not register it."[1], pgs 2-4
---
On the odd chance that somehow the Copyright Office had reversed itself I then went back to part 2 of the report:
"As the Office affirmed in the Guidance, copyright protection in the United States
requires human authorship. This foundational principle is based on the Copyright Clause in
the Constitution and the language of the Copyright Act as interpreted by the courts. The
Copyright Clause grants Congress the authority to “secur[e] for limited times to authors . . . the exclusive right to their . . . writings.” As the Supreme Court has explained, “the author [of a copyrighted work] is . . . the person who translates an idea into a fixed, tangible expression entitled to copyright protection.”
"No court has recognized copyright in material created by non-humans, and those that
have spoken on this issue have rejected the possibility. "
"In most cases, however, humans will be involved in the creation process, and the work
will be copyrightable to the extent that their contributions qualify as authorship." -- [2], pgs 15-16
---
TL;DR If you make something with the assistance of AI, you still have to be personally involved and contribute more than just a prompt in order to receive copyright, and then you will receive protection only over such elements of originality and authorship that you are responsible for, not those elements which the AI is responsible for.
You should care. If someone submits a huge PR, you’re going to waste time asking questions and comprehending their intentions if the answer is that they don’t know either. If you know it’s generated and they haven’t reviewed it themselves, you can decide to shove it back into an LLM for next steps rather than expect the contributor to be able to do anything with your review feedback.
Unreviewed generated PRs can still be helpful starting points for further LLM work if they achieve desired results. But close reading with consideration of authorial intent, giving detailed comments, and asking questions from someone who didn't write or read the code is a waste of your time.
That's why we need to know if a contribution was generated or not.
You are absolutely right. AI is just a tool to DDoS maintainers.
Any contributor who was shown to post provably untested patches used to lose credibility. And now we're talking about accommodating people who don't even understand how the patch is supposed to work?
That’s not what I said though. LLM output, even unreviewed and without understanding, can be a useful artifact. I do it all the time - generate code, try running it, and then if I see it works well, I can decide to review it and follow up with necessary refactoring before integrating it. Parts of that can be contributed too. We’re just learning new etiquettes for doing that productively, and that does includes testing the PR btw (even if the code itself is not understood or reviewed).
If the AI slop was that valuable a project regular, who actually knows and understands the project, would be just as capable of asking the AI to produce it.
Not according to ghostty maintainer Hashimoto per above.
It takes attempts, verifying the result behaves as desired, and iterative prompting to adjust. And it takes a lot of time to wait on agents in between those steps (this work isn’t a one shot response). You’re being reductive.
We may be talking cross purposes. I read the grandparent poster discussing provably untested patches.
I have no clue in ghostty but I've seen plenty of stuff that doesn't compile much less pass tests. And I assert there is nothing but negative value in such "contributions".
If real effort went into it, then maybe there is value-- though it's not clear to me: When a project regular does the same work then at least they know the process. Like if there is some big PR moving things around at least the author knows that it's unlikely to slip in a backdoor. Once the change is reduced to some huge diff, it's much harder to gain this confidence.
In some projects direct PRs for programmatic mass renames and such have been prohibited in favor of requiring submission of the script that produces the change, because its easier to review the script carefully. The same may be necessary for AI.
Having the original prompts (in sequence and across potentially multiple models) can be valuable but is not necessarily useful in replicating the results because of the slot machine nature of it
> This whole original HN post is about ghostty btw
Sure though I believe few commenters care much about ghostty specifically and are primarily discussing the policy abstractly!
> because of the slot machine nature of it
One could use deterministically sampled LLMs with exact integer arithmetic... There is nothing fundamental preventing it from being completely reproducible.
Can't do that with state of the art LLMs and no sign of that changing (as they like to retain control over model behaviors). I would not want to use or contribute to a project that embraces LLMs yet disallows leading models.
Besides, the output of an LLM is not really any more trustworthy (even if reproducible) than the contribution of an anonymous actor. Both require review of outputs. Reproducibility of output from prompt doesn't mean that the output followed a traceable logic such that you can skip a full manual code review as with your mass renaming example. LLMs produce antagonistic output from innocuous prompting from time to time, too.
It would be nice if they did, in fact, say they didn't know. But more often they just waste your time making their chatbot argue with you. And the chatbots are outrageous gaslighters.
All big OSS projects have had the occasional bullshitter/gaslighter show up. But LLMs have increased the incidence level of these sorts of contributors by many orders of magnitude-- I consider it an open question if open-public-contribution opensource is viable in the world post LLM.
There was some post that comes to mind of an example of this. Some project had a security issue reported that was not a security issue, and when asking questions it became extremely obvious that someone was just feeding the conversation into an LLM. There was no security issue. I can imagine this is happening more and more as people are trying to slam in LLM generated code everywhere.
Everyone promoting LLMs, especially on HN, claim that they're expertly using them by using artisanal prompts and carefully examining the output but.. I'm honestly skeptical. Sure, some people are doing that (I do it from time to time). But I've seen enough slop to think that more people are throwing around code that they barely understand than these advocates care to admit .
Those same people will swear that they did due diligence, but why would they admit otherwise? And do they even know what proper due diligence is? And would they still be getting their mythical 30%-50% productivity boost if they were actually doing what they claimed they were doing?
And that is a problem. I cannot have a productive code review with someone that does not even understand what their code is actually doing, much less trade offs that were made in an implementation (because they did not consider any trade offs at all and just took what the LLM produced). If they can't have a conversation about the code at all because they didn't bother to read or understand anything about it, then theres nothing I can do except close the PR and tell them to actually do the work this time.
Agreed. As someone who uses AI (completion and Claude Code), I'll disclose whenever asked. But I disagree that it's "common courtesy" when not explicitly asked; since many people (including myself) don't mind and probably assume some AI, and it adds distraction (another useless small indicator; vaguely like dependabot, in that it steals my attention but ultimately I don't care).
That's nonsense. It's like feeling you need to disclose that your IDE has autocomplete. Nobody discloses that, since it's ridiculous. You only disclose that you used Claude Code if you are not certain of the result (e.g. you think it is correct, but the maintainer might be a better judge).
If it's exactly the same as what you'd have written manually, and you are confident it works, then what's the point of disclosure?
It’s completely different from an IDE’s autocomplete because autocomplete in an IDE is only helping you type identifiers that already exist in your codebase or in any SDKs you’re using.
An LLM is regurgitating things from outside that space, where you have no idea of the provenance of what it’s putting into your code.
It doesn’t just matter that the code you’re contributing to a project is correct, it matters quite a lot if it’s actually something you’re allowed to contribute.
- You can’t contribute code that your employer owns to a project if they don’t want you to.
- You can’t contribute code under a license that the project doesn’t want you to use.
- And you can’t contribute code written by someone else and claim it’s your intellectual property without some sort of contract in place to grant that.
If you use an LLM to generate code that you’re contributing, you have both of the latter two problems. And all of those apply *even if* the code you’re contributing is identical to what you’d have written by hand off the top of your head.
When you contribute to a project, you’re not just sending that project a set of bits, you’re making attestations about how those bits were created.
Why does this seem so difficult for some supposed tech professionals to understand? The entire industry is intellectual property, and this is basic “IP 101” stuff.
> Why does this seem so difficult for some supposed tech professionals to understand?
Maybe because 99% of people that complain about this complain about problems that never occur in 99% of the cases they cite. My employer isn’t going to give a shit that code that I’ve written for their internal CRUD app gets more or less directly copied into my own. There’s only one way to do that, it was already in my head before I wrote it for them, and it’ll still be in after. As long as I’m not directly competing with their interests, what the hell do they care.
> When you contribute to a project, you’re not just sending that project a set of bits, you’re making attestations about how those bits were created.
You are really not. You are only doing that if the project requires some attestation of provenance. I can tell you that none of mine do.
No one is forcing you to read it. Feel free to have your own AI judge if you should merge it or even just YOLO merge it. The end goal of people trying to get code merged is not to have you read it. It's to improve the software. Whether code improves the software or not is orthogonal to if the code was written by hand.
The reason it's common courtesy is out of respect for the reviewer/maintainer's time. You need to let em know to look for the kind of idiotic mistakes LLMs shit out on a routine basis. It's not a "distraction", it's extremely relevant information. On the maintainer's discretion, they may not want to waste their time reviewing it at all, and politely or impolitely ask the contributor to do it again, and use their own brain this time. It also informs them on how seriously to take this contributor in the future, if the work doesn't hold water, or indeed, even if it does, since the next time the contributor runs the LLM lottery the result may be utter bullshit.
Whether it's prose or code, when informed something is entirely or partially AI generated, it completely changes the way I read it. I have to question every part of it now, no matter how intuitive or "no one could get this wrong"ish it might seem. And when I do, I usually find a multitude of minor or major problems. Doesn't matter how "state of the art" the LLM that shat it out was. They're still there. The only thing that ever changed in my experience is that problems become trickier to spot. Because these things are bullshit generators. All they're getting better at is disguising the bullshit.
I'm sure I'll gets lots of responses trying to nitpick my comment apart. "You're holding it wrong", bla bla bla. I really don't care anymore. Don't waste your time. I won't engage with any of it.
I used to think it was undeserved that we programmers called ourselved "engineers" and "architects" even before LLMs. At this point, it's completely farcical.
"Gee, why would I volunteer that my work came from a bullshit generator? How is that relevant to anything?" What a world.
FWIW, I can say from direct experience people that other people are watching and noting when people are submitting AI slop as their own work, and taking note to never hire these people. Beyond the general professional ethics, it makes you harder to distinguish from malicious parties and other incompetent people LARPing as having knowledge that they don't.
then it's not clear why you would have to disclose talking to an AI.
Generally speaking, when someone uses the word "slop" when talking about AI it's a signal to me that they've been sucked into a culture war and to discount what they say about AI.
It's of course the maintainer's right to take part in a culture war, but it's a useful way to filter out who's paying attention vs who's playing for a team. Like when you meet someone at a party and they bring up some politician you've barely heard of but who their team has vilified.
> then it's not clear why you would have to disclose talking to an AI.
It’s explained right there in the PR:
> The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
That is not true of books, search engines, stack overflow, or talking to a worker, because in all those cases you still had to do the work yourself of comprehending, preparing, and submitting the patch. This is also why they ask for a disclosure of “the extent to which AI assistance was used”. What about that isn’t clear to you?
The reality is as someone that helps maintain several OSS projects you vastly underestimate the problem that AI-assisted tooling has created.
On the one hand, it's lowered the barrier to entry for certain types of contributions. But on the other hand getting a vibe-coded 1k LOC diff from someone that has absolutely no idea how the project even works is a serious problem because the iteration cycle of getting feedback + correctly implementing it is far worse in this case.
Also, the types of errors introduced tend to be quite different between humans and AI tools.
It's a small ask but a useful one to disclose how AI was used.
When one side has much more "scalability" than the other, then the other side has very strong motivation to match up.
- People use AI to write cover letters. If the companies don't filter out them automatically, they're screwed.
- Companies use AI to interview candidates. No one wants to spend their personal time talking to a robot. So the candidates start using AI to take interviews for them.
etc.
If you don't at least tell yourself that you don't allow AI PRs (even just as a white lie) you'll one day use AI to review PRs.
Both sides will use AI and it will ultimately increase economic productivity.
Imagine living before the invention of the printing press, and then lamenting that we should ban them because it makes it "too easy" to distribute information and will enable "low quality" publications to have more reach. Actually, this exact thing happened, but the end result was it massively disrupted the world and economy in extremely positive ways.
Compilers don’t randomly fail to compile code that is too difficult for them to understand. Llvm makes sure that I never have to learn assembly, gpt doesn’t guarantee at all that I don’t have to learn to code.
Ironically, I thought your parent commenter had to go through mental gymnastics to say that their parents analogy of the printing press isn't applicable to an LLM. Neither you nor your parent gave me any satisfactory reasons why they aren't similar, just your mental superiority as proof that oceanplexian must be wrong.
> Imagine living before the invention of the printing press, and then lamenting that we should ban them because it makes it "too easy" to distribute information
Imagine seeing “rm -rf / is a function that returns “Hello World!” and thinking “this is the same thing as the printing press”
> Both sides will use AI and it will ultimately increase economic productivity.
In some cases sure but it can also create the situation where people just waste time for nothing (think AI interviewing other AIs - this might generate GDP by people purchasing those services but I think we can all agree that this scenario is just wasting time and resource without improving society).
Imagine being so deluded and disconnected that you actually believe that AI has any similarity with the printing press regarding the benefits to The People.
Whether the output of AI can be copyrighted remains a legal minefield, so if I were running a project where copyright-based protections are important (say, anything GPL) I would want to know if a PR contained them.
If a maintainer asks me to jump through too many stupid hoops, I'll just not contribute to the software.
That said, requiring adequate disclosure of AI is just fair. It also suggests that the other side is willing to accept AI-supported contributions (without being willing to review endless AI slop that they could have generated themselves if they had the time to read it).
I would expect such a maintainer to respond fairly to "I first vibecoded it. I then made manual changes, vibecoded a test, cursorily reviewed the code, checked that the tests provide good coverage, ran both existing and new tests, and manually tested the code."
That fair response might be a thorough review, or a request that I do the thorough review before they put in the time, but I'd expect it to be more than a blatant "nope, AI touched this, go away".
I won't put it as "just another tool". AI introduces a new kind of tool where the ownership of the resulting code is not straightforward.
If, in the dystopian future, a justice court you're subjected to decides that Claude was trained on Oracle's code, and all Claude users are possibly in breach of copyright, it's easier to nuke from orbit all disclosed AI contributions.
As a project maintainer, you shouldn't make rules unenforceable rules that you and everyone else know people will flout. Doing so comes makes you seem impotent and diminishes the respect people have for rules in general.
You might argue that by making rules, even futile ones, you at least establish expectations and take a moral stance. Well, you can make a statement without dressing it up as a rule. But you don't get to be sanctimonious that way I guess.
Except you can enforce this rule some of the time. People discover that AI was used or suspect it all the time, and people admit to it after some pressure all the time.
Not every time, but sometimes. The threat of being caught isn't meaningless. You can decide not to play in someone else's walled garden if you want but the least you can do is respect their rules, bare minimum of human decency.
The only legitimate reason to make a rule is to produce some outcome. If your rule does not result in that outcome, of what use is the rule?
Will this rule result in people disclosing "AI" (whatever that means) contributions? Will it mitigate some kind of risk to the project? Will it lighten maintainer load?
No. It can't. People are going to use the tools anyway. You can't tell. You can't stop them. The only outcome you'll get out of a rule like this is making people incrementally less honest.
You’re basically saying “if a rule can be broken, it will be, therefore rules are useless.”
If someone really wants to commit fraud they’re going to commit fraud. (For example, by not disclosing AI use when a repository requires it.) But if their fraud is discovered, they can still be punished for it, and mitigating actions taken. That’s not nothing, and does actually do a lot to prevent people from engaging in such fraud in the first place.
There are plenty of argumentative and opinionated reasons to say it matters, but there is one that can't really be denied - reviewers (and project maintainers, even if they aren't reviewers) are people whose time deserves to be respected.
If this rule discourages low quality PRs or allows reviewers to save time by prioritizing some non-AI-generated PRs, then it certainly seems useful in my opinion.
Yes that is the stated purpose, did you read the linked GitHub comment? The author lays out their points pretty well, you sound unreasonably upset about this. Are you submitting a lot of AI slop PRs or something?
P.S Talking. Like. This. Is. Really. Ineffective. It. Makes. Me. Just. Want. To. Disregard. Your. Point. Out. Of. Hand.
The utility of the rule is so that you can cheaply nuke non-conforming contributors from orbit when you detect their undisclosed AI use. Vs having to deal with the flood of low quality contributions on a individually reviewed basis.
Unenforceable rules are bad, but if you tweak the rule to always require some sort of authorship statement (e.g. "I wrote this by hand" or "I wrote this with Claude"), then the honor system will mostly achieve the desired goal of calibrating code review effort.
> As a project maintainer, you shouldn't make rules unenforceable rules
Total bullshit. It's totally fine to declare intent.
You are already incapable of verifying / enforcing that a contributor is legally permitted to submit a piece of code as their own creation (Signed-off-by), and do so under the project's license. You won't embark on looking for prior art, for the "actual origin" of the code, whatever. You just make them promise, and then take their word for it.
And if they’re discovered to not be keeping their word, there can be consequences imposed and mitigating actions taken. Rules can’t prevent bad actions 100% of the time, but they can substantially increase the risk of bad actions.
We keep talking about “AI replacing coders,” but the real shift might be that coding itself stops looking like coding. If prompts become the de facto way to create applications/developing systems in the future, maybe programming languages will just be baggage we’ll need to unlearn.
Programming languages were a nice abstraction to accommodate our inability to comprehend complexity - current day LLMs do not have the same limitations as us.
The uncomfortable part will be what happens to PRs and other human-in-the-loop checks. It’s worthwhile to consider that not too far into the future, we might not be debugging code anymore - we’ll be debugging the AI itself. That’s a whole different problem space that will need an entirely new class of solutions and tools.
This fundamentally misunderstands why programming languages exist. They're not required because "we can't understand complexity". They were invented because we need a way to be very specific about what we want the machine to do. Whether it's the actual physical hardware we're talking to when writing assembly, or it's an abstract machine that will be translated to the hardware like in C or Java, the key point is that we want to be specific.
Natural language can be specific, but it requires far too many words. `map (+ 1) xs` is far shorter to write than "return a list of elements by applying a function that adds one to its argument to each element of xs and collecting the results in a separate list", or similar.
Fair enough - I’m just the messenger though, observing the current trends and extrapolating from there. Let’s talk about “AGENTS.md” files quickly. We’re specifying what I consider “rules” in plain language. Even lint rules (instead of creating a lint config). Could be a matter of convenience, and if it gets us 80% of the way, why not?
I believe it won’t be long before we have exceptional “programmers” who have mastered the art of vibe coding. If that does become the de facto standard for 80% programming done, then it’s not a long stretch from there that we might skip programming languages altogether. I’m simply suggesting that if you’re not going to examine the code, perhaps someone will eliminate that additional layer or step altogether, and we might be pleasantly surprised by the final result.
Good idea! We can have some sort of standard grammar that we use to prompt the LLM such that it deterministically gives us the result we ask for. We then constrain all prompts to match that grammar. Some sort of language describing programs.
How does this not lead to a situation where no honest person can use any AI in their submissions? Surely pull requests that acknowledge AI tooling will be given significantly less attention, on the grounds that no one wants to read work that they know is written by AI.
I don't think this is the case. Mitchell writes that he himself uses LLMs, so it's not black and white. A PR author who has a deep understanding of their changes and used an LLM for convenience will be able to convey this without losing credibility imo
I doubt a PR is going to be buried if it's useful, well designed, good code, etc, just because of this disclosure. Articulate how you used AI and I think you've met the author's intent.
If the PR has issues and requires more than superficial re-work to be acceptable, the authors don't want to spend time debugging code spit out by an AI tool. They're more willing to spend a cycle or two if the benefit is you learning (either generally as a dev or becoming more familiar with the project). If you can make clear that you created or understand the code end to end, then they're more likely to be willing to take these extra steps.
Seems pretty straightforward to me and thoughtful by the maintainers here.
> I doubt a PR is going to be buried if it's useful, well designed, good code, etc, just because of this disclosure
If that were the case, why would this rule be necessary, if it indeed is the substance that matters? AI generated anything has a heavy slop stigma right now, even if the content is solid.
This would make for an interesting experiment to submit a PR that was absolute gold but with the disclaimer it was generated with help of ChatGPT. I would almost guarantee it would be received with skepticism and dismissals.
The rule is necessary because the maintainers want to build good will with contributors and if a contributor makes a bad PR but could still learn from it then they will put effort into it. It's a "if you made a good effort, we'll give you a good effort" and using AI tools gives you a very low floor for what "effort" is.
If you make a PR where you just used AI, it seems to work, but didn't go further then the maintainers can go "well I had a look, it looks bad, you didn't put effort in, I'm not going to coach you through this". But if you make a PR where you go "I used AI to learn about X then tried to implement X myself with AI writing some of it" then the maintainers can go "well this PR doesn't look good quality but looks like you tried, we can give some good feedback but still reject it".
In a world without AI, if they were getting a lot of PRs from people who obviously didn't spend any time on their PRs then maybe they would have a "tell us how long this change took you" disclosure as well.
> While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
If it's bad code from a person he'll help them get it fixed. If it's bad code from an AI why bother?
The last three GitHub issues I ran across when looking something up had people literally copy pasting the entire ChatGPT response as their comment. It feels like I'm living in some crazy dystopia when several _different_ people post a 30+ line message that's 95% the same. I'm sorry but I refuse to interact with people who do this, if I wanted to talk to a computer I'd do it myself.
It just might. But if people generate a bias against AI generated code because AI can generate massive amounts of vaguely correct looking yet ultimately bad code then that seems like an AI problem not a people problem. Get better, AI coding tools.
Submitting a PR also means you’re not submitting code copied from elsewhere without calling that out and ensuring license compatibility, we don’t refer to that as incentivizing lying about the origin of submitted code.
Fraud and misrepresentation are always options for contributors, at some point one needs to trust that they’re adhering to the rules that they agreed to adhere to.
It might encourage people to be dishonest, or to not contribute at all. Maybe that's fine for now, but what if the next generation come to rely on these tools?
> Do companies want to hire "honest" people whose CVs were written by some LLM?
Yes, some companies do want to hire such people, the justification given is something along the lines of "we need devs who are using the latest tools/up to date on the latest trends! They will help bring in those techniques and make all of our current devs more productive!". This isn't a bad set of motivations or assumptions IMO.
Setting aside what companies _want_, they almost certainly are already hiring devs with llm-edited CVs, whether they want it or not. Such CVs/resumes are more likely to make it through HR filters.
I don't know if future generations will agree with this sentiment, in which case we lock ourselves out of future talent (i.e. those that use AI to assist, not to completely generate). The same arguments were made about Photoshop once upon a time.
> Do companies want to hire "honest" people whose CVs were written by some LLM?
Unfortunately yes, they very much seem to. Since many are using LLMs to assess CVs, those which use LLMs to help write their CV have a measured advantage.
This is such an important conversation. At Delve, we use AI extensively to automate compliance workflows, but we're always transparent about it with our customers.
The real question isn't whether AI was used, but whether it adds genuine value.
Does the AI actually solve a problem? Is it making people's lives easier? Can you explain how it works?
The best AI tools feel invisible - they just make complex tasks simple. The worst ones make you constantly aware you're dealing with AI because they're clunky or unreliable.
Transparency builds trust. If your AI tooling is genuinely helpful, you should be proud to talk about it.
Helps the maintainer focus their efforts during review. If you wrote that exact sentence, a maintainer would keep "understanding of the codebase" as a place for potential sources of incorrect assumptions higher in mind than otherwise.
In my experience, if there's one thing the AI assistants do really well, it's understanding existing code and summarizing it or pinpointing specific things I'm looking for.
I'm not really a developer but I routinely need to go dig through codebases looking for answers and AI assistants have made this so much faster. This has been a boon for me.
I like the pattern of including each prompt used to make a given PR, yes, I know that LLM's aren't deterministic, but it also gives context of the steps required to get to the end state.
I’m using specsytory in vscode + cursor for this - it keeps a nice little md doc of all your LLM interactions, and you can check that into source control if you like so it’s included in pull requests, and can be referenced during code review.
I've been doing that for most of the commits in this project as an experiment, gemini, human, or both. Not sure what I'm going to do with that history, but I did at least want to start capturing it
It's ridiculous and impractical, honestly. A single AI-generated PR would likely involve at least 10-20 prompts, interspersed with testing, manual edits to context / guideline files without which those prompts don't have the same effect, manual coding, and more. A screen recording would do better.
And if contributions are that unwelcome, then it's better not to contribute. There has to be some baseline level of trust that the contributor is trying to do the right thing; I get enough spying from the corporations in my life.
I'm not talking screen recorder but a log file where I could be given it and use it for input and repeat the work exactly as it was done. Sort of like how one could do the same with their bash history file to repeat an initially exploratory analysis effort using the same exact commands. I'm surprised that isn't already a capability given the business interest with AI. One would think they would like to cache these prompt flows to ensure long term fidelity in process.
Keep in mind that in industries where people code but aren't really programmers this literally does happen, sometimes very "big" people will be basically scared to share their code because it won't be very good.
But anyway what I mean is that code is us speaking like a computer, LLMs are the other way around, you can see a lot from how someone interacts with the machine.
not just non-programmers. It's a common problem with jr programmers and good programmer internships (for example) make it a point of forcing the interns to expose themselves ASAP to get over it.
I think if everyone goes into it knowing that it'll be part of what they publish it would be less of an issue.
I mean, unless you're all a bunch of freaks who have instructed your LLM to cosplay as Slave Leia and can't work otherwise, in which case your issues are beyond my pay grade. :P
I still do not understand how one can integrate "AI" code into a project with a license at all. "AI" code is not copyrightable, "AI" cannot sign a contributor agreement.
So if the code is integrated, the license of the project lies about parts of the code.
The contributor is the human that chose to run the LLM, not the “AI” itself - so the real question is, why isn’t the human’s code copyrightable, and why can’t the human sign a contributor agreement?
Besides, this stuff is not what the author is concerned about:
> I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code … I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort.
They want to coach aspiring contributors based on code they’ve written themselves, not based on how they prompt their AI.
It’s a matter of how they enjoy spending their time.
Your question makes sense. See U.S. Copyright Office publication:
> If a work's traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it.
> For example, when an AI technology receives solely a prompt from a human and produces complex written, visual, or musical works in response, the “traditional elements of authorship” are determined and executed by the technology—not the human user...
> For example, if a user instructs a text-generating technology to “write a poem about copyright law in the style of William Shakespeare,” she can expect the system to generate text that is recognizable as a poem, mentions copyright, and resembles Shakespeare's style. But the technology will decide the rhyming pattern, the words in each line, and the structure of the text.
> When an AI technology determines the expressive elements of its output, the generated material is not the product of human authorship. As a result, that material is not protected by copyright and must be disclaimed in a registration application.
> In other cases, however, a work containing AI-generated material will also contain sufficient human authorship to support a copyright claim. For example, a human may select or arrange AI-generated material in a sufficiently creative way that “the resulting work as a whole constitutes an original work of authorship.”
> Or an artist may modify material originally generated by AI technology to such a degree that the modifications meet the standard for copyright protection. In these cases, copyright will only protect the human-authored aspects of the work, which are “independent of” and do “not affect” the copyright status of the AI-generated material itself.
> This policy does not mean that technological tools cannot be part of the creative process. Authors have long used such tools to create their works or to recast, transform, or adapt their expressive authorship. For example, a visual artist who uses Adobe Photoshop to edit an image remains the author of the modified image, and a musical artist may use effects such as guitar pedals when creating a sound recording. In each case, what matters is the extent to which the human had creative control over the work's expression and “actually formed” the traditional elements of authorship.
In any but a pathological case, a real contribution code to a real project has sufficient human authorship to be copyrightable.
> the license of the project lies about parts of the code
That was a concern pre-AI too! E.g. copy-past from StackOverflow. Projects require contributors to sign CLAs, which doesn't guarantee compliance, but strengthens the legal position. Usually something like:
"You represent that your contribution is either your original creation or you have sufficient rights to submit it."
I don’t see much benefit from the disclosure alone. Ultimately, this is code that needs to be reviewed. There is going to continue to be more and more AI assisted code generation, to the point where we see the same level of adoption of these tools as "Autocomplete". Why not solve this through tooling? I have had great effect with tools like Greptile, Cursor's BugBot and Claude Code.
And with a better and more useful response. Instead of wasting time on the technical details, you can give feedback like "this isn't the sort of change that AI is likely to be helpful with, though if you want to keep trying make at least sure your PRs pass the tests." or "If you'd like to share your prompts we might be able to make some suggestions, we've found on this project it's useful to include <X>".
Sure it needs to be reviewed. But the author does more than just reviewing, they help the person submitting the PR to improve their PR. If the other side is an AI, it can save them some time.
It's a spectrum, isn't it? I wouldn't want to waste my time reviewing a bunch of repetitive code generated from some script or do something like review every generated template instantiation in a C++ code base. I would want to review the script/template definition/etc., but what's the equivalent for AI? Should the review just be the prompt(s)?
Edit: Also, it's always good to provide maximal context to reviewers. For example, when I use code from StackOverflow I link the relevant answer in a comment so the reviewer doesn't have to re-tread the same ground I covered looking for that solution. It also gives reviewers some clues about my understanding of the problem. How is AI different in this regard?
I always appreciated Claude Code's commit authoring. Whereas I think a lot of people were offended that "their" work was being overshadowed by an AI's signature.
We probably will do similar in the Bun repo. We use AI tools a lot and don’t discourage it, but we also label when a PR was generated by claude both via github label and also by leaving in the “Generated with claude code” textual watermark.
Heh, that's a great way to make a point, but right now AI is nowhere near what a traditional editor autocomplete is. Yes you can use it that way, but it's by no means limited to that. If you think of AI as a fancy autocomplete, that's a good personal philosophy, but there are plenty of people that aren't using it that way
This seems misguided. The human submitter is accountable for the code submitted. Doesn’t matter if the AI built it, a contractor in another timezone, or ten thousand monkeys typing away.
Github provides a profile but it’s not meant to make this kind of assessment. Basically a trust badge/metrics of some sort you could check before engaging with someone’s PR.
1. The world has fundamentally changed due to LLMs. You don't know where a code submission falls between "written thoroughly with eternal vigilance" vs "completely vibe-coded" since it's now trivially to generate the later. There's no going back. And a lot of comments here seem stuck on this point.
2. The maintainer naively or stubbornly imagines that he can get everyone to pre-sort their code between the two buckets through self-reporting.
But that's futile.
It's like asking someone if they're a good person on a date because you don't want to waste your time with bad people. Unfortunately, that shortcut doesn't exist.
Now, maybe going forward we will be forced to come up with real solutions to the general problem of vetting people. But TFA feels like more of a stunt than a serious pitch.
> maybe going forward we will be forced to come up with real solutions to the general problem of vetting people
The monekeys paw closes a finger and now you need a formal certification, professional license, and liability insurance to publish software or source code.
It's not about "feeling agency" or fabricating a justification. As the PR says:
"AI tooling must be disclosed for contributions
I think, at this stage of AI, it is a common courtesy to disclose this.
In a perfect world, AI assistance would produce equal or higher quality work than any human. That isn't the world we live in today, and in many cases it's generating slop. I say this despite being a fan of and using them successfully myself (with heavy supervision)! I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was.
The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code."
I don't think it has anything to do with being a reactionary movement or counter culture. If it were I would expect, among other things, that it would absolutely prohibit the use of AI entirely rather than just require disclosure.
The background is that many higher profile open source projects are getting deluged by low quality AI slop "contributions", not just crappy code but when you ask questions about it you sometimes get an argumentative chatbot lying to you about what the PR does.
And this latest turn has happened on top of other trends in 'social' open source development that already had many developers considering adopting far less inclusive practice. RETURN TO CATHEDRAL, if you will.
The problem isn't limited to open source, it's also inundating discussion forums.
Yeah, but this is once again doing the thing where you read someone replying to situation #2 (in my comment) and then acting like they are denying situation #1.
I think we all grant that LLM slop is inundating everything. But a checkbox that says "I am human" is more of a performative stunt (which I think they are referring to when they say it's reactionary) than anything practical.
Cloudflare's "I am human" checkbox doesn't just take your word for it, and imagine if it did.
---
People who write good, thoughtful code and also use an LLM have no reason to disclose it just because the idea offends someone, just like I don't disclose when I do an amphetamine bender to catch up in work; I don't want to deal with any prejudices someone might have, but I know I do good work and that's what matters. I pressed tab so the LLM could autocomplete the unit test for me because it's similar to the other 20 unit tests, and then I vetted the completion. I'm not going to roleplay with anyone that I did something bad or dishonest or sloppy here; I'm too experienced to know better.
People who write slop that they can't even bother to read themselves aren't going to bother to read your rules either. Yet that's the group OP is pretending he's going to stop. Once you get past the "rah-rah LLM sux0rs amirite fellow HNers?" commentary, there's nothing here.
What value does being snotty and dismissive have? they're just going to copy and paste your reply to their chatbot. The toaster doesn't have feelings you can hurt.
So the nice thing about open source is you get to make your own rules. But then you need to accept the consequences.
For example, the GNU project has certain norms, and those dissuade a lot of people from contributing (e.g. I prefer working on projects with simpler, non-viral licenses). The limited volunteer labor pool is allocated according to people's interests towards other projects, and maybe GNU projects suffer for less attention on them.
I just submitted my first big open source contribution to the OpenAI agents SDK for JS. Every word except the issue I opened was done by AI.
On the flip side, I’m preparing to open source a project I made for a serializable state machine with runtime hooks. But that’s blood sweat and tears labor. AI is writing a lot of the unit tests and the code, but it’s entirely by my architectural design.
There’s a continuum here. It’s not binary. How can we communicate what role AI played?
And does it really matter anymore?
(Disclaimer: autocorrect corrected my spelling mistakes. Sent from iPhone.)
I think its simple, just don't hide it. I've had mutliple contributors try to hide the fact that they used AI (E.g removing claude as a code author - they didn't know how to do it and close the PR when it first happened.). I don't really care if someone uses AI, but most of the people who do also do not test their changes which just gives me more work. If someone:
1.) Didn't try to hide the fact that they used AI
2.) Tested their changes
I would not care at all. The main issue is this is usually not the case, most people submitting PRs that are 90% AI do not bother testing (Usually they don't even bother running the automated tests)
Well, if you had read what was linked, you would find these...
> I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was.
> The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
> I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code.
I don't know specifically what PR's this person is seeing. I do know it's been a rumble around the open source community that inexperienced devs are trying to get accepted PRs for open source projects because they look good on a resume. This predated AI in fact, with it being a commonly cited method to get attention in a competitive recruiting market.
As always, folks trying to get work have my sympathies. However ultimately these folks are demanding time and work from others, for free, to improve their career prospects while putting in the absolute bare minimum of effort one could conceivably put in (having Copilot rewrite whatever part of an open source project and shove it into a PR with an explanation of what it did) and I don't blame them for being annoyed at the number of low-quality submissions.
I have never once criticized a developer for being inexperienced. It is what it is, we all started somewhere. However if a dev generated shit code and shoved it into my project and demanded a headpat for it so he could get work elsewhere, I'd tell him to get bent too.
The OP seems to be coming from the perspective of "my time as a PR reviewer is limited and valuable, so I don't want to spend it coaching an AI agent or a thin human interface to an AI agent". From that perspective, it makes perfect sense to want to know how much a human is actually in the loop for a given PR. If the PR is good enough to not need much review then whether AI wrote it is less important.
An angle not mentioned in the OP is copyright - depending on your jurisdiction, AI-generated text can't be copyrighted, which could call into question whether you can enforce your open source license anymore if the majority of the codebase was AI-generated with little human intervention.
As long as some of the code is written by humans it should be enforceable. If we assume AI code has no copyright (not sure it has been tested in courts yet) then it would only be the parts written by the AI. So if AI writes 100 lines of code in Ghostty then I guess yes someone can "steal" that code (but no other code in Ghostty). Why would anyone do that? 100 random lines of AI code in isolation isn't really worth anything...
- For ages now, people have used "broad test coverage" and "CI" as excuses for superficial reviews, as excuses for negligent coding and verification.
- And now people foist even writing the test suite off on AI.
Don't you see that this way you have no reasoned examination of the code?
> ... and the code, but it’s entirely by my architectural design.
This is fucking bullshit. The devil is in the details, always. The most care and the closest supervision must be precisely where the rubber meets the road. I wouldn't want to drive a car that you "architecturally designed", and a statistical language model manufactured.
if you read his note i think he gives good insight as to why he wants PRs to signal AI involvement.
that being said i feel like this is an intermediate step - it's really hard to review PRs that are AI slop because it's so easy for those who don't know how to use AI to create a multi-hundred/thousand line diff. but when AI is used well, it really saves time and often creates high quality work
As long as they make it easy to add a “made with AI” tag to the PR, it seems like there’s really no downside. I personally can’t imagine why someone would want to hide the fact they used AI. A contractor would not try to hide that they used an excavator to dig a hole instead of a shovel.
I guess if you write 1000 lines and you just auto tabbed an auto-complete of a variable name done by AI you might not wanna say the code is written by AI.
>I personally can’t imagine why someone would want to hide the fact they used AI.
Because of the perception that anything touched by AI must be uncreative slop made without effort. In the case of this article, why else are they asking for disclosure if not to filter and dismiss such contributions?
Did you actually read the post? The author describes exactly why. It's not to filter and dismiss, but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't actually understand anyway. If you can articulate how you used AI, demonstrate that you understand the problem and your proposed solution (even if AI helped get you there), then I'm sure the maintainers will be happy to work with you to get a PR merged.
>I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
>but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't
This is very much in line with my comment about doing it to filter and dismiss. The author didn't say "So I can reach out and see if their clear eagerness to contribute extends to learning to code in more detail".
There's a lot of discussion about the ethics of AI disclosure going on here, but this maintainer is writing this from a place I recognize - community-driven, participatory development by people who enjoy programming, talking about it, and teaching each other how to do it.
Not every software project needs to attempt to maximize productivity. Not every software project is a business. Some are just created by people who enjoy programming. By hand. It's OK to set that as a culture. I guess I get it.
I don't mind AI tools, I use them judiciously, but sometimes I just want to do some coding - for me it's a genuinely relaxing and mentally satisfying activity to write some good code -, and I'm happy there's still others around me who do as well. Gardening context prompts and what not just isn't nearly as fun as just doing it, and not every project has to be economical. This one is yet another terminal emulator; it's not going to be the next unicorn. But I bet it's fun to hack on.
This is actually a well worded, reasonable suggestion that rightly draws attention to the lack of review and informed oversight of AI-assisted contributions in some cases.
It's ironically in rather a contrast in tone to some of the anti-AI comments in the thread here that seem to be responding to tired arguments from three years ago rather than the OP linked.
Little offtop: would someone remember mitchellh's setup for working with AI tools? I remember someone posted in an AI-hate-love threads here and it's not in the his blog[1]
Blaming it on the tool, and not the person's misusing it trying to get his name on a big os project, is like blaming the new automatic in the kitchen and not the chef for getting a raw pizza on the table.
OP is not blaming the AI - did you read the post? AI does enable shitty humans to open PRs with code they have no comprehension of, wasting precious time donated by a skilled maintainer. That is a new thing that wasn’t possible without AI.
Using AI to generate code in a PR does not necessarily mean however that the user has not taken time to understand the changes and is not willing to learn. There are AI users who generate whole files without understanding the contents, and then there are AI users who generate the exact same files but have known in advance what they want, and merely use AI as a tool to save typing.
The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway.
Sorry, I can't see how your comment is a reply to my comment. I said AI enables shitty humans to do this. I didn't say everyone using AI is a shitty human or whatever it is you took away from that.
> The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway.
That is a fair way to see it, and I agree that it is a losing battle if your battle is enforcing this rule.
However, from a different perspective - if one sees it more as a gentlemen agreement (which it de facto is) - it fosters an environment where like-minded folks can cooperate better.
The disclosure assists the reviewer in finding common issues in AI generated code, the specificity of the disclosure even more so.
For example, a submitter sends a PR where they disclose a substantial amount of the code was AI assisted but all tests were manually written. The disclosure allows the reviewer to first look at the tests to gauge how well the submitter understands and constrained the solution to their requirements, the next step being to then look at the solution from a high-level perspective before going into the details. It respects the reviewer time, not necessarily because the reviewer is above AI usage, but because without disclosure the whole collaborative process falls apart.
Not sure how long this can work though, it's still easy to distinguish bad code written by a human from AI slop. In the first case your review and assistance is an investment into the collaborative process, in the latter it's just some unread text included in the next prompt.
I think that in the FOSS environmment, it is assumed that when you submit something upstream, that you are the copyright holder. Some projects like GNU require you to sign papers legally attesting this.
It would be a lie to sign those papers for something you vibe coded.
It's not just courtesy; you are committing fraud if you put your copyright notice on something you didn't create and publishing that to the world.
I don't just want that disclosed; I cannot merge it if it is disclosed, period.
I know that purely AI-generated content is copyright-free, but I don't think that AI-assisted is also copyright free.
If I use iOS's spellchecker which "learns" from one's habit (i.e.: AI, the really polished kind), I don't lose copyright over the text which I've written.
If AI told you have a missing semicolon and it regenerated an almost exact copy of your code, with only the added semicolon being different, then the case is very strong that the fixed code is a derived work of only your code. (Moreover, side point: you could burn that session, keeping the fix, and nobody would ever suspect.)
If it's purely generated, then there is no question it's a derived work of unknown works used without permission.
Those are the extreme endpoints. I think that the bulk of the intermediate area between steers toward infringment; the contribution from AI has to be very low or trivial to have a clear case that the work is still clean.
This is because a small amount of illegal material taints the whole work; it's not a linear interpolation whereby if you wrote 90% of it, it's 90% in the clear. E.g. one GPL-ed source file in a work with 50,000 source files requires the whole thing to be GPLed, or else that file removed.
> If you are using any kind of AI assistance while contributing to Ghostty,
*this must be disclosed in the pull request*, along with the extent to
which AI assistance was used (e.g. docs only vs. code generation).
If PR responses are being generated by an AI, disclose that as well.
The extent here is very important. There's a massive difference between vibe-coding a PR, using LLMs selectively to generate code in files in-editor, and edit prediction like copilot.
It says actually later that tab-completion needn't be disclosed.
What if you did not use a prompt, but, for example, the fancy auto-complete feature in Visual Studio that for example GitHub Copilot provides (i.e. accept huge code completion automatically suggested by Copilot via tab key)?
I think ghostty is a popular enough project that it attracts a lot of attention, and that means it certainly attracts a larger than normal amount of interlopers. There are all kinds of bothersome people in this world, but some of the most bothersome you will find are well meaning people who are trying to be helpful.
I would guess that many (if not most) of the people attempting to contribute AI generated code are legitimately trying to help.
People who are genuinely trying to be helpful can often become deeply offended if you reject their help, especially if you admonish them. They will feel like the reprimand is unwarranted, considering the public shaming to be an injury to their reputation and pride. This is most especially the case when they feel they have followed the rules.
For this reason, if one is to accept help, the rules must be clearly laid out from the beginning. If the ghostty team wants to call out "slop", then it must make it clear that contributing "slop" may result in a reprimand. Then the bothersome want-to-be helpful contributors cannot claim injury.
This policy, and most of the comments on here, miss the point entirely.
Open source is supposed to be primarily a labor of love, scratching your own itch, done for pride and a sense of accomplishment of having done it right.
Using an AI chatbot achieves none of that. It's an admission that you don't have the skills, and you're not wanting to learn them. Using AI to contribute to an open source project makes NO sense whatsoever. It's an admission of incompetence, which should be profoundly embarrassing to you.
I realize I'm screaming into the void here on HN, where so many people have bought into the "AI is just a tool" bullshit. But I know I'm not alone, even here. And I know that e.g. Linus Torvalds sees right through the bullshit as well.
Hot take: if you can't spot any issues in the code review it's either good code, code that needs further changes, or review was not done properly. I don't see how "I used LLMs" fit here, because it means nothing to the quality of the code submitted.
If such mention would mean increased reviewer attention, then every code review should include it.
I contribute to an open-source game with decades of legacy and employing many unusual patterns.
Two weeks ago someone asked me to review a PR, which I did pointing out some architectural concerns. The code was not necessarily bad, but it added boilerplate to a point it required some restructuring to keep the codebase maintainable. I couldn't care less if it was written by AI or not in this case, it was not necessarily simple but was straightforward.
It took considerable effort to conjure a thoughtful and approachable description of the problem and the possible solutions. Keep in mind this is a project with considerable traction and a high amount of contributions are from enthusiasts. So having a clear, maintainable and approachable codebase is sometimes the most important requirement.
They asked for a second pass but it took two weeks for me to get around it, in the meantime they sent 3 different PRs, one closed after the other. I found it a bit strange, then I put some effort to review the last iteration. It had half baked solutions where for example there would be state cleanup functions but state was never written in the first place, something that would never go through normally, I gave the benefit of the doubt and pointed it out. I suspected it was AI generated most likely.
Then they showed me another variation of the PR where they implement a whole different architecture, some incredibly overengineered fluent interface to resolve a simple problem with many layers of indirection that reflects complete indifference to the more nuanced considerations relevant to the domain and project, that I tried to impair. The code might work, but even if it does it's obvious that the change is a net negative to the project.
What I suspected indeed was the case, as they finally disclosed the use of AI, but that is not necessarily the problem, as I hope to convey. The problem is that I was unable to gauge the submitters commitment to perform the humble job of _understanding_ what I proposed. The proposal, in the end, just becoming mere tokens for inclusion into a prompt. Disclosure wouldn't necessarily have caused me to not take the PR seriously, instead I would invested my time in the more productive effort of orienting the submitter on the tractability of achieving their goal.
I would rather have known they didn't intend or gauged their capacity beforehand. It would have been better for both of us: they would have had their initial iteration merged (which was fine, I would just have shrugged the refactor for another occasion) and I wouldn't have lost time.
Your story highlights the clash of two mindsets regarding open source development. On one side there are artisans who treat programming as a craft and care about every line of code. On the other side, vibe-coders who might not even seen the code LLMs have generated and don't care about it as long as the program works.
Some context: it is a tightly knit community, where I interact with the contributors often in a conversational manner.
I know their intent is to push the project forward and well-meaning, I don't care about whether they are vibe-coding. I care about knowing they are vibe-coding so I can assist them to vibe-code in a way they can actually achieve their goal, or help them realize early that they lack the capacity to contribute (not of their own fault necessarily, maybe they just require orientation on how to reason about problems and solutions or their use of the tools).
Agreed -- this requirement feels less like an actually useful requirement and more a silly and performative one, which is trying to make some kind of commentary on AI use as a whole.
Spending at minimum five minutes would tell you why maintainers are implementing this change. It's because people using LLMs are spamming open source repos with fake issues, incredibly low quality but high effort to review PRs and shutting down the active communication process between reviewer and reviewee by not even understanding their own code.
Why would these people disclose their use of AI? These are not responsible and thoughful users of AI. The slop producers won't disclose, and the responsible users who produce high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point.
> Why would these people disclose their use of AI?
Because lying about your usage of AI is a good way to get completely kicked out of the open source community once caught. That's like asking 'why should you bother with anti-cheating measures for speedruns'. Why should we have any guidelines or regulations if people are going to bypass them? The answer I hope should be very obvious.
> high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point.
Then obviously the repository in question doesn't want people using AI and you should go elsewhere. They're not even against LLM tooling for this repo but people are freaking out because how dare you ask me to disclose what tools I'm using.
Nothing stopping these low effort contributors from lying and saying no LLMs were used - especially in the world of AI slop where such contributions are not welcome.
Doesn't anyone see that this can't be policed or everyone becomes a criminal? That AI will bring the end of copyrights and patents as we know them when literally everything becomes a derivative work? When children produce better solutions than industry veterans so we punish them rather than questioning the divine right of corporations to rule over us? What happened to standing on the shoulders of giants?
I wonder if a lot of you are as humbled as I am by the arrival of AI. Whenever I use it, I'm in awe of what it comes up with when provided almost no context, coming in cold to something that I've been mulling over for hours. And it's only getting better. In 3-5 years, it will leave all of us behind. I'm saying that as someone who's done this for decades and has been down rabbit holes that most people have no frame of reference for.
My prediction is that like with everything, we'll bungle this. Those of you with big egos and large hoards of wealth that you thought you earned because you are clever will do everything you can to micromanage and sabotage what could have been the first viable path to liberation from labor in human history. Just like with the chilling effects of the Grand Upright Music ruling and the DMCA and HBO suing BitTorrent users (edit: I meant the MPAA and RIAA, HBO "only" got people's internet shut off), we have to remain eternally vigilant or the powers that be will take away all the fun:
So no, I won't even entertain the notion of demanding proof of origin for ideas. I'm not going down this slippery slope of suing every open source project that gives away its code for free, just because a PR put pieces together in a new way but someone thought of the same idea in private and thinks they're special.
In the age of LLMs responsible maintainers should treat contributor PRs the same way they treat AI slop: rewrite it if it's salvageable and ignore it if it's not. Accepting any contribution 'as is' shouldn't be allowed unless it's a trivial 1-liner (with a thorough investigation attached).
The emergent trait of AI laceing every other sentence/comment with emojis is a pretty good signal for when you want to ignore AI slop. I almost wish it was codified in to the models.
I'll cover in my YouTube why this is wrong but TLDR: you need to evaluate quality not process. AI can be used in diametrically different ways and the reason why this policy could be enforced is because it will be obvious if the code is produced via a solo flight of some AI agent. For the same reason that's not a policy that will improve anything.
If you look at how Quality Assurance works everywhere outside of software it is 99.9999% about having a process which produces quality by construction.
Respectfully, Mr. Redis, sir, that's what's going on. I don't see any reason to make a video about it. From the PR that's TFA:
"In a perfect world, AI assistance would produce equal or higher quality work than any human. That isn't the world we live in today, and in many cases it's generating slop. I say this despite being a fan of and using them successfully myself (with heavy supervision)! I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was.
The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code."
"I don't see any reason to make a video about it". This sentence is so wrong in its depth that's difficult to know where to start arguing against it.
"The disclosure is to help maintainers assess how much attention to give a PR."
For the same reason we should ask how many years contributing have been writing software and in the specific language, as they are also correlated with quality of produced code.
"we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code"
Yes, producing great code and documentation, regardless the process.
Provenance matters. An LLM cannot certify a Developer Certificate of Origin (https://en.wikipedia.org/wiki/Developer_Certificate_of_Origi...) and a developer of integrity cannot certify the DCO for code emitted by an LLM, certainly not an LLM trained on code of unknown provenance. It is well-known that LLMs sometimes produce verbatim or near-verbatim copies of their training data, most of which cannot be used without attribution (and may have more onerous license requirements). It is also well-known that they don't "understand" semantics: they never make changes for the right reason.
We don't yet know how courts will rule on cases like Does v Github (https://githubcopilotlitigation.com/case-updates.html). LLM-based systems are not even capable of practicing clean-room design (https://en.wikipedia.org/wiki/Clean_room_design). For a maintainer to accept code generated by an LLM is to put the entire community at risk, as well as to endorse a power structure that mocks consent.
For a large LLM I think the science in the end will demonstrate that verbatim reproduction is not coming from verbatim recording, as the structure really isn’t setup that way in the models under question here.
This is similar to the ruling by Alsup in the Anthropic books case that the training is “exceedingly transformative”. I would expect a reinterpretation or disagreement on this front from another case to be both problematic and likely eventually overturned.
I don’t actually think provenance is a problem on the axis you suggest if Alsups ruling holds. That said I don’t think that’s the only copyright issue afoot - the copyright office writing on copyrightability of outputs from the machine essentially requires that the output fails the Feist tests for human copyrightability.
More interesting to me is how this might realign the notion of copyrightability of human works further as time goes on, moving from every trivial derivative bit of trash potentially being copyrightable to some stronger notion of, to follow the feist test, independence and creativity. Further it raises a fairly immediate question in an open source setting if many individual small patch contributions themselves actually even pass those tests - they may well not, although the general guidance is to set the bar low - but is a typo fix either? There is so far to go on this rabbit hole.
I'd be fine with that if that was the way copyright law had been applied to humans for the last 30+ years but it's not. Look into the OP's link on clean room reverse engineering, I come from an RE background and people are terrified of accidentally absorbing "tainted" information through extremely indirect means because it can potentially used against them in court.
I swear the ML community is able to rapidly change their mind as to whether "training" an AI is comparable to human cognition based on whichever one is beneficial to them at any given instant.
So if you can get an LLM to produce music lyrics, for example, or sections from a book, those would be considered novel works given the encoding as well?
Depends if the music is represented by the RIAA or not :)
"an LLM" could imply an LLM of any size, for sufficiently small or focused training sets an LLM may not be transformative. There is some scale at which the volume and diversity of training data and intricacy of abstraction moves away from something you could reasonably consider solely memorization - there's a separate issue of reproduction though.
"novel" here depends on what you mean. Could an LLM produce output that is unique that both it and no one else has seen before, possibly yes. Could that output have perceived or emotional value to people, sure. Related challenge: Is a random encryption key generated by a csprng novel?
In the case of the US copyright office, if there wasn't sufficient human involvement in the production then the output is not copyrightable and how "novel" it is does not matter - but that doesn't necessarily impact a prior production by a human that is (whether a copy or not). Novel also only matters in a subset of the many fractured areas of copyright laws affecting the space of this form of digital replication. The copyright office wrote: https://www.copyright.gov/ai/Copyright-and-Artificial-Intell....
Where I imagine this approximately ends up is some set of tests that are oriented around how relevant to the whole the "copy" is, that is, it may not matter whether the method of production involved "copying", but may more matter if the whole works in which it is included are at large a copy, or, if the area contested as a copy, if it could be replaced with something novel, and it is a small enough piece of the whole, then it may not be able to meet some bar of material value to the whole to be relevant - that there is no harmful infringement, or similarly could cross into some notion of fair use.
I don't see much sanity in a world where small snippets become an issue. I think if models were regularly producing thousands of tokens of exactly duplicate content that's probably an issue.
I've not seen evidence of the latter outside of research that very deliberately performs active search for high probability cases (such as building suffix tree indices over training sets then searching for outputs based on guidance from the index). That's very different from arbitrary work prompts doing the same, and the models have various defensive trainings and wrappings attempting to further minimize reproductive behavior. On the one hand you have research metrics like 3.6 bits per parameter of recoverable input, on the other hand that represents a very small slice of the training set, and many such reproductions requiring strongly crafted and long prompts - meaning that for arbitrary real world interaction the chance of large scale overlap is small.
By novel, I mean if I ask a model to write some lyrics or code and it produces pre-existing code or lyrics, is it novel and legally safe to use because the pre-existing code or lyrics aren’t precisely encoded in a large enough model, and therefore legally not a reproduction just coincidentally identical.
No. I don't think "novelty" would be relevant in such a case. How much risk you have depends on many factors, including what you mean by "use". If you mean sell, and you're successful, you're at risk. That would be true even if it's not the same as other content but just similar. Copyright provides little to no protection from legal costs if someone is motivated to bring a case at you.
In the West you are free to make something that everyone thinks is a “derivative piece of trash” and still call it yours; and sometimes it will turn out to be a hit because, well, it turns out that in real life no one can reliably tell what is and what isn’t trash[0]—if it was possible, art as we know it would not exist. Sometimes what is trash to you is a cult experimental track to me, because people are different.
On that note, I am not sure why creators in so many industries are sitting around while they are being more or less ripped off by massive corporations, when music has got it right.
— Do you want to make a cover song? Go ahead. You can even copyright it! The original composer still gets paid.
— Do you want to make a transformative derivative work (change the composition, really alter the style, edit the lyrics)? Go ahead, just damn better make sure you license it first. …and you can copyright your derivative work, too. …and the original composer still gets credit in your copyright.
The current wave of LLM-induced AI hype really made the tech crowd bend itself in knots trying to paint this as an unsolvable problem that requires IP abuse, or not a problem because it’s all mostly “derivative bits of trash” (at least the bits they don’t like, anyway), argue in courts how it’s transformative, etc., while the most straightforward solution keeps staring them in the face. The only problem is that this solution does not scale, and if there’s anything the industry in which “Do Things That Don’t Scale” is the title of a hit essay hates then that would be doing things that don’t scale.
[0] It should be clarified that if art is considered (as I do) fundamentally a mechanism of self-expression then there is, of course, no trash and the whole point is moot.
There's an whole genre of musicians focusing only on creating royalty free covers of popular songs so the music can be used in suggestive ways while avoiding royalties.
It's not art. It's parasitism of art.
> There's an whole genre of musicians focusing only on creating royalty free covers
There is no such thing as a “royalty free cover”. Either it is a full on faithful cover, which you can perform as long as license fees are paid, and in which case both the performer and the original songwriter get royalties, or it is a “transformative cover” which requires negotiation with the publisher/rights owner (and in that case IP ownership will probably be split between songwriter and performer depending on their agreement).
(Not an IP lawyer myself so someone can correct me.)
Furthermore, in countries where I know how it works as a venue owner you pay the rights organization a fixed sum per month or year and you are good to go and play any track you want. It thus makes no difference to you whether you play the original or a cover.
Have you considered that it is simply singers-performers who like to sing and would like to earn a bit of money from it, but don’t have many original songs if their own?
> It's parasitism of art
If we assume covers are parasitism of art, by that logic would your comment, which is very similar to dozens I have seen on this topic in recent months, be parasitism of discourse?
Jokes aside, a significant number of covers I have heard at cafes over years are actually quite decent, and I would certainly not call that parasitic in any way.
Even pretending they were, if you compare between artists specialising in covers and big tech trying to expropriate IP, insert itself as a middleman and arbiter for information access, devalue art for profit, etc., I am not sure they are even close in terms of the scale of parasitism.
> Have you considered that it is simply singers-performers who like to sing and would like to earn a bit of money from it, but don’t have many original songs if their own?
Or, maybe you start to pay attention?
They are selling their songs cheaper for TV, radio or ads.
> Even pretending they were, if you compare between artists specialising in covers and big tech trying to expropriate IP
They're literally working for spotify.
> They are selling their songs cheaper for TV, radio or ads.
I guess that somehow refutes the points I made, I just can’t see how.
Radio stations, like the aforementioned venue owners, pay the rights organizations a flat annual fee. TV programs do need to license these songs (as unlike simple cover here the use is substantially transformative), but again: 1) it does not rip off songwriters (holder of songwriter rights for a song gets royalties for performance of its covers, songwriter has a say in any such licensing agreement), and 2) often a cover is a specifically considered and selected choice: it can be miles better fitting for a scene than the original (just remember Motion Picture Soundtrack in that Westworld scene), and unlike the original it does not tend to make the scene all about itself so much. It feels like you are yet to demonstrate how it is particularly parasitic.
Edit: I mean honest covers; modifying a song a little bit and passing it as original should be very sueable by the rights holder and I would be very surprised if Spotify decided to do that even if they fired their entire legal department and replaced it with one LLM chatbot.
I know of restaurants and bars that choose to play cover versions of well-known songs because the costs are so much less.
I really doubt you would ever license any specific songs as a cafe business. You should be able to pay a fixed fee to a PRO and have a blanket license to play almost anything. Is it so expensive in the US, or perhaps they do not know that this is an option? If the former, and those cover artists help those bars keep their expenses low and offer you better experience while charging less—working with the system, without ripping off the original artists who still get paid their royalty—does it seem particularly parasitic?
The example I was referring to was not in the US.
A restaurant / cafe may pay a fixed fee and get access to a specific catalog of songs (performances). The fee depends on what the catalog contains. As you can imagine, paying for the right to only play instrumental versions of songs (no singers, no lyrics) is significantly cheaper. Or, having performances of songs by unknown people.
Two countries where I know how it works from a venue business owner perspective work this way. The fees seemed pretty mild, that’s why I asked if it’s too expensive in your country (which I guess is not US).
There's several sides of music copyright:
1. The lyrics
2. The composition
3. The recording
These can all be owned by different people or the same person. The "royalty free covers" you mention are people abusing the rights of one of those. They're not avoiding royalties, they just havn't been caught yet.
I believe performance of a cover still results in relevant royalties paid to the original songwriter, just sans the performance fee, which does not strike me as a terrible ripoff (after all, a cover did take effort to arrange and perform).
What this person is talking about is they write “tvinkle tvinkle ittle stawr” instead of the real lyrics (basically just writing the words phonetically and/or misspelled) to try and bypass the law through “technicalities” that wouldn’t stand up in court.
I doubt so for a few reasons based on how they described this alleged parasitic activity, but mainly because the commenter alluded to Spotify doing this. Would be very surprising if they decided to do something so blatantly illegal, when they could keep extracting money by the truckload with their regular shady shenanigans that do not cross that legality line so obviously.
Regarding what you described, I don’t think I encountered this in the wild enough to remember. IANAL but if not cleared/registered properly as a cover it doesn’t seem to be a workaround or abuse, but would probably be found straight up illegal if the rights holder or relevant rights organization cares to sue. In this case, all I can say is “yes, some people do illegal stuff”. The system largely works.
> For a large LLM I think the science in the end will demonstrate that verbatim reproduction is not coming from verbatim recording
We don't need all this (seemingly pretty good) analysis. We already know what everyone thinks: no relevant AI company has had their codebase or other IP scraped by AI bots they don't control, and there's no way they'd allow that to happen, because they don't want an AI bot they don't control to reproduce their IP without constraint. But they'll turn right around and be like, "for the sake of the future, we have to ingest all data... except no one can ingest our data, of course". :rolleyes:
There are only so many ways to code quite a few things. My classmate and I once got in trouble in high school for having identical code for one of the tasks at a coding competition, down to variable names and indentation. There is no way he could or would steal my code, and I sure didn't steal his.
This is how sqlite handles it,
> Contributed Code
> In order to keep SQLite completely free and unencumbered by copyright, the project does not accept patches. If you would like to suggest a change and you include a patch as a proof-of-concept, that would be great. However, please do not be offended if we rewrite your patch from scratch.
source, https://www.sqlite.org/copyright.html
Or you know, they just feel like code should be free. Like beer should be free.
We didn't have this whole issue 20 years ago because nobody gave a shit. If your code was public, and on the internet, it was free for everyone to use by definition.
An LLM can be used for a clean room design so long as all (ALL) of its training data is in the clean room (and consequently does not contain the copyrighted work being reverse engineered).
An LLM trained on the Internet-at-large is also presumably suitable for a clean room design if it can be shown that its training completed prior to the existence of the work being duplicated, and thus could not have been contaminated.
This doesn't detract from the core of your point, that LLM output may be copyright-contaminated by LLM training data. Yes, but that doesn't necessarily mean that an LLM output cannot be a valid clean-room reverse engineer.
> An LLM trained on the Internet-at-large is also presumably suitable for a clean room design if it can be shown that its training completed prior to the existence of the work being duplicated, and thus could not have been contaminated.
This is assuming that you are only concerned with a particular work when you need to be sure that you are not copying any work that might be copyrighted without making sure to have a valid license that you are abiding by.
The "clean room" in "clean room reverse engineering" refers to a particular set of trade secrets, yes. You could have a clean room and still infringe if an employee in the room copied any work they had ever seen.
The clean room has to do with licenses and trade secrets, not copyright.
There is also IP taint when using "AI". We're just pretending that there's not.
If someone came to you and said "good news: I memorized the code of all the open source projects in this space, and can regurgitate it on command", you would be smart to ban them from working on code at your company.
But with "AI", we make up a bunch of rationalizations. ("I'm doing AI agentic generative AI workflow boilerplate 10x gettin it done AI did I say AI yet!")
And we pretend the person never said that they're just loosely laundering GPL and other code in a way that rightly would be existentially toxic to an IP-based company.
Courts (at least in the US) have already ruled that use of ingested data for training is transformative. There’s lots of details to figure, but the genie is out of the bottle.
Sure it’s a big hill to climb in rethinking IP laws to align with a societal desire that generating IP continue to be a viable economic work product, but that is what’s necessary.
> Courts (at least in the US) have already ruled that use of ingested data for training is transformative
Yes, the training of the model itself is (or should be) a transformative act so you can train a model on whatever you have legal access to view.
However, that doesn't mean that the output of the model is automatically not infringing. If the model is prompted to create a copy of some copyrighted work, that is (or should be) still a violation.
Just like memorizing a book isn't infringment but reproducing a book from memory is.
The fact that GitHub’s Copilot has an enterprise feature that matches model output against code having certain licenses - in order to prevent you from using it, with a notification - suggests the model outputs are at least potentially infringing.
If MS were compelled to reveal how these completions are generated, there’s at least a possibility that they directly use public repositories to source text chunks that their “model” suggested were relevant (quoted as it could be more than just a model, like vector or search databases or some other orchestration across multiple workloads).
> suggests the model outputs are at least potentially infringing.
The only thing it suggests is that they recognize that a subset of users worry about it. Whether or not GitHub worries about it any further isn’t suggested.
Don’t think about it from an actual “rights” perspective. Think about the entire copyright issue as a “too big to fail” issue.
> directly use public repositories
I don't see why a company which has been waging a multi decade war against GPL and users' rights would stop at _public_ repositories.
This also matches my (not a lawyer) intuition, but have there been any legal precedents set in this direction yet?
Some courts at some levels. It’s by no means settled law.
Most licensed software is unsettled law, if we're being that pedantic.
Not really, no. If you’re specifically referring to, say, GPL or BSD or other Open Source licenses, it’s a bit more unsettled, but software licensing as a whole has several decades of case law at this point.
> Courts (at least in the US) have already ruled that use of ingested data for training is transformative
This is far from settled law. Let's not mischaracterize it.
Even so, an AI regurgitating proprietary code that's licensed in some other way is a very real risk.
No more so than regurgitating an entire book. While it could technically be possible in the case of certain repos that are ubiquitous on the internet (and therefore overrepresented in training data to the point that they are "regurgitated" verbatim, in whole), it is extremely unlikely and would only occur after deliberate prompting. The NYT suit against Open AI shows (in discovery) that the NYT was only able to get partial results after deliberately prompting the model with portions of the text they were trying to force it to regurgitate.
So. Yes, technically possible. But impossible by accident. Furthermore when you make this argument you reveal that you don't understand how these models work. They do not simply compress all the data they were trained on into a tiny storable version. They are effectively multiplication matrices that allow math to be done to predict the most likely next token (read: 2-3 Unicode characters) given some input.
So the model does not "contain" code. It "contains" a way of doing calculations for predicting what text comes next.
Finally, let's say that it is possible that the model does spit out not entire works, but a handful of lines of code that appear in some codebase.
This does not constitute copyright infringement, as the lines in question a) represent a tiny portion of the whole work (and copyright only protecst against the reduplication of whole works or siginficant portions of the work), and B) there are a limited number of ways to accomplish a certain function and it is not only possible but inevitable that two devs working independently could arrive at the same implementation. Therefore using an identical implementation (which is what this case would be) of a part of a work is no more illegal than the use of a certain chord progression or melodic phrasing or drum rhythm. Courts have ruled about this thoroughly.
> No more so than regurgitating an entire book.
Like this?
Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book - https://news.ycombinator.com/context?id=44972296 - 67 days ago (313 comments)
Yes, that is one of those works that is over-represented in the training data, as I explained in the part of the comment you clearly did not comprehend.
> you clearly did not comprehend
I comprehend it just fine, I was adding context for those who may not comprehend.
It's also why some companies do clean room design.
> Sure it’s a big hill to climb in rethinking IP laws to align with a societal desire that generating IP continue to be a viable economic work product, but that is what’s necessary.
Well, AI can perhaps solve the problem it created here: generated IP with AI is much cheaper than with humans, so it will be viable even at lower payoffs.
Less cynical: you can use trade secrets to protect your IP. You can host your software and only let customers interact with it remotely, like what Google (mostly) does.
Of course, this is a very software-centric view. You can't 'protect' eg books or music in this way.
In the US you can not generate copyrightable IP without substantial human contribution to the process.
https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...
Eh, they figured out how to copyright photographs, where the human only provides a few bits (setting up the scene, deciding when to pull the trigger etc); so stretching a few bits of human input to cover the whole output of an AI should also be doable with sufficiently well paid lawyers.
It's already been done.
https://en.wikipedia.org/wiki/Generative_artificial_intellig...
Tell that to Reddit. They’re AI translating user posts and serving it up as separate Google search results. I don’t remember if Reddit claims copyright on user-submitted content, or on its AI translations, but I don’t think Reddit is paying ad share like X is, either, so it kind of doesn’t matter to the user, as they’re (still) not getting paid, even as Reddit collects money for every ad shown/clicked. Even if OP did write it, an AI translated the version shown.
https://news.ycombinator.com/context?id=44972296
reddit is a user hostile company, have been forever, always will be. they take rights over your content, farm things about you, sell data, do invasive things in the mobile apps, use creepware cookies, etc.
Excerpt from the user agreement:
People put their heads in the sand over reddit for some reason, but it's worse than FAANG.Interesting. Apparently you don't waive any moral rights, unlike at Reddit. That is, you should still get credited for your work (in theory).
To a certain reading, this is user-centric: it’s increasing the size of the audience pool beyond that of shared language speakers and readers to the entire literate human race. This is an important point to acknowledge, because every silver lining has its cloud.
User-centric would be giving users choice.
It really is that simple.
Forcing something on people from a position of power is never in their favor.
I don’t think having a Reddit account is mandatory.
As a user of Reddit, I think it’s cool, and also raises some concerns.
I think most sites that handle user data are going to have rough edges. Making money off of user content is never without issues.
It's not about being mandatory. It's about having a privileged position and using it to extract value from people.
The nature of network effects is such that once a site gets as big as reddit (or facebook or tiktok or whichever), it's nearly impossible for competition to take over in the same design space.
Many communities (both small and large) are only present on specific platforms (sometimes only one) and if you want to participate you have to accept their terms or exclude yourself socially.
If they have a privileged position, it is earned and freely given. No one is obligated to use the site. The issue is more one of the commons being enclosed and encircled by corporate interests, and then branded and commodified. Once the deed is done, there is no reason for folks to leave, because everyone they know is there.
Most communities on Reddit that I’d care to be a part of have additional places to gather, but I do take your point that there are few good alternatives to r/jailbreak, for example.
The host always sets its own rules. How else could anything actually get done? The coordination problem is hard enough as it is. It’s a wonder that social media exists at all.
Gatekeepers will always exist adjacent to the point of entry, otherwise every site turns extremist and becomes overrun with scammers and spammers.
An AI model's output can be transformative, but you can be unlucky enough that the LLM memorized the data that it gave you.
I don't see why verbatim or not should matter at all.
How complex does a mechanical transformation have to be to not be considered plagiarism, copyright infringement or parasitism?
If somebody writes a GPL-licensed program, is it enough to change all variable and function names to get rid of those pesky users' rights? Do you have to change the order of functions? Do you have to convert it to a different language? Surely nobody would claim c2rust is transformative even though the resulting code can be wildly different if you apply enough mechanical transformations.
All LLMs do is make the mechanical transformations 1) probabilistic 2) opaque 3) all at once 4) using multiple projects as a source.
> How complex does a mechanical transformation have to be to not be considered plagiarism, copyright infringement or parasitism?
Legally speaking, this depends from domain to domain. But consider for example extracting facts from several biology textbooks, and then delivering those facts to the user in the characteristic ChatGPT tone that is distinguishable from the style of each source textbook. You can then be quite assured that courts will not find that you have infringed on copyright.
> Courts (at least in the US) have already ruled that use of ingested data for training is transformative.
If you have code that happens to be identical to some else's code or implements someone's proprietary algorithm, you're going to lose in court even if you claim an "AI" gave it to you.
AI is training on private Github repos and coughing them up. I've had it regurgitate a very well written piece of code to do a particular computational geometry algorithm. It presented perfect, idiomatic Python with perfect tests that caught all the degenerate cases. That was obviously proprietary code--no amount of searching came up with anything even remotely close (it's why I asked the AI, after all).
>If you have code that happens to be identical to some else's code or implements someone's proprietary algorithm, you're going to lose in court even if you claim an "AI" gave it to you.
Not for a dozen lines here or there, even if it could be found and identified in a massive code base. That’s like quoting a paragraph of a book in another book, non infringing.
For the second half of your comment it sounds like you’re saying you got results that were too good to be AI- that’s a bit “no true Scotsman”, at least without more detail. But implementing an algorithm, even a complex one, is very much something an LLM can do. Algorithms are much better defined and scoped natural language, and LLMs do a reasonable job of translating to languages. An algorithm is a narrow subset of that task type with better defined context and syntax.
> Not for a dozen lines here or there, even if it could be found and identified in a massive code base. That’s like quoting a paragraph of a book in another book, non infringing.
It's potentially non-infringing in a book if you quote it in a plausible way, and properly.
If you copy&paste a paragraph from another book into yours, it's infringing, and a career-ending scandal. There's plenty of precedent on that.
Just like if you manually copied a function out of some GPL code and pasted it into your own.
Or if you had an LLM do it for you.
What will happen when company A implements algorithm X based on AI output, company B does the same and company A claims that it is proprietary code and takes company B to court?
What has happened when the same thing happens without AI involved?
Yep, it’s not a brand-new problem. I just wonder if AI is going to turbocharge the odds of these disputes popping up.
How is that obviously proprietary? Aren't you implicitly assuming that the AI couldn't have written it on its own?
It cannot do anything on its own, it's just a (very complex, probabilistic) mechanical transformation (including interpolation) of training data and a prompt.
Advertising autocomplete as AI was a genius move because people start humanizing it and look for human-centric patterns.
Thinking A"I" can do anything on its own is like seeing faces in rocks on Mars.
The idea that something that can't handle simple algorithms (e.g. counting the number of times a letter occurs in a word) could magically churn out far more advanced algorithms complete with tests is… well it's a bit of a stretch.
It's terrible at executing algorithms. This, it turns out, is completely disjoint from writing algorithms.
This makes no sense. Computational geometry algorithms are computable.
LLMs aren't good at rote memorization. They can't even get quotations of humans right.
It's easier for the LLM to rewrite an idiomatic computational geometry algorithm from scratch in a language it understands well like Python. Entire computational geometry textbooks and research papers are in its knowledge base. It doesn't have to copy some proprietary implementation.
A search for "LLM Harry Potter" would suggest that LLMs are widely understood to be proficient at rote memorization.
(I find the example of the computational geometry algorithm being a clear case of direct memorization not very compelling, in any case.)
That seems a real stretch. GPT 5 just invented new math for reference. What you are saying would be equivalent to saying that this math was obviously in some paper that mathematician did not know about. Maybe true, but it's a far reach.
It invented "new math" as much as I invented "new food" when I was cooking yesterday. It did a series of quite complicated calculations that would take a well trained human several hours or even days to do - still impressive, but no it's not new maths.
An example: https://medium.com/@deshmukhpratik931/the-matrix-multiplicat...
Obviously not ChatGPT. But ChatGPT isn't the sharpest stick on the block by a significant margin. It is a mistake to judge what AIs can do based on what ChatGPT does.
New math? As in it just fucking Isaac Newton'd invented calculus? Or do you just mean it solved a math problem?
This would be the first time ever that an LLM has discovered new knowledge, but the far reach is that the information does appear in the training data?
They've been doing it for a while. Gemini has also discovered new math and new algorithms.
There is an entire research field of scientific discovery using LLMs together with sub-disciplines for the various specialization. LLMs routinely discover new things.
I hadn't heard of that, so I did some searching and the single source for the claim I can find is a Google white paper. That doesn't automatically mean it's false, of course, but it is curious that the only people ostensibly showing LLMs discover new things are the companies offering the LLMs.
Citation needed, and I call bullshit. Unless you mean that they hallucinate useless algorithms that do not work, which they do.
LLMs do not have an internal model for manipulating mathematical objects. They cannot, by design, come up with new algorithms unless they are very nearly the same as some other algorithm. I'm a computer science researcher and have not heard of a single algorithm created by LLM.
https://medium.com/@deshmukhpratik931/the-matrix-multiplicat...
And it's not an accident that significant percentage (40%?) of all papers being published in top journals involve application of AIs.
This article is about the same thing I mentioned in a sibling comment. I personally don't find an unreplicated Google white paper to be compelling evidence.
It's a fast matrix multiply! (A decades-old human problem). What exactly do you need to replicate??! Just count the multiplies, fer goodness sake.
> What exactly do you need to replicate??!
The AI coming up with it? When Google claimed their Wizard of Oz show at the Las Vegas Sphere was AI-generated, a ton of VFX artists spoke up to say they'd spent months of human labor working on it. Forgive me for not giving the benefit of the doubt to a company that has a vested interest in making their AI seem more powerful, and a track record of lying to do so.
>societal desire that generating IP continue to be a viable economic work product
It is strange that you think the law is settled when I don't think even this "societal desire" is completely settled just yet.
Maybe I should clarify: Society, in general, supports the idea that writers, artists, film makers, coders, etc— everyone who creates IP- should have a place in the economy. Basically just that it should be possible to make a living and have a career at it. It can be spun different ways, and those differences are important, but this is the basic thing.
This doesn’t seem like a disputable statement to me. For anyone who thinks actors’ likenesses, authors’ words, all of it- that all and everything should be up for grabs once written or put anywhere in public, that is not a widely held opinion.
Once that’s established, it all comes down to implementation details.
> Society, in general, supports the idea that writers, artists, film makers, coders, etc
Coders don't get paid every single time their code runs. Why bundle different rights together?
> Coders don't get paid every single time their code runs.
They do if they code the API correctly.
> Why bundle different rights together?
Why are mineral rights sold separately to most land deeds?
> Why are mineral rights sold separately to most land deeds?
Because the population does not rebel against the politicians that made these laws.
That’s a matter of contract and licensing, not the limits of copyright law.
Training an AI model is not the same as using an AI model.
If you mean the ruling has absolutely no applicability when it comes to using the model then, no, that is incorrect:
Judge Alsup, in his ruling, specifically likened the process to reading text and then using the knowledge to write something else. That’s training and use.
How are you verifying that the AI is writing "something else" and not something that's verbatum in its training material?
Publishing pirated copies of books on libgen isn't the same as downloading pirated copies of books from libgen. Neither is legal.
In many places that's a fairly recent development: publishing pirated IP used to be much more of a legal problem than consuming it.
Also publishing pirated IP without any monetary gain to yourself also used to be treated more leniently.
Of course, all the rules were changed (both in law and in interpretation in practice) as file sharing became a huge deal about two decades ago.
Details depend on jurisdiction.
I’m curious … So “transformative” is not necessarily “derivative”?
Seems to me the training of AI is not radically different than compression algorithms building up a dictionary and compressing data.
Yet nobody calls JPEG compression “transformative”.
Could one do lossy compression over billions of copyrighted images to “train” a dictionary?
> I’m curious … So “transformative” is not necessarily “derivative”?
(not legal advice)
Transformative works are necessarily derivative, but that transformation allows for a legal claim to "fair use" regardless of making a derived work.
https://en.wikipedia.org/wiki/Transformative_use
A compression algorithm doesn't transform the data it stores it in a different format. Storing a story in a txt file vs word file doesn't transform the data.
An llm is looking at the shape of words and ideas over scale and using that to provide answers.
No a compression algorithm does transform the data, particularly lossy ones. The pixels stored in the output are not in the input, they're new pixels. That's why you can't uncompress a jpeg. Its a new image that just happens to look like the original. But it even might not - some jpegs are so deep fried they become their own form of art. This is very popular in meme culture.
The only difference, really, is we know how a JPEG algorithm works. If I wanted to, I could painstakingly make a jpeg by hand. We don't know how LLMs work.
Also ban StackOverflow and nearly any text book in the field.
The reality is that programmers are going to see other programmers code.
"see" and "copy" are two different things. It's fine to look at StackOverflow to understand the solution to a problem. It's not fine to copy and paste from StackOverflow and ignore its license or attribution.
Content on StackOverflow is under CC-by-sa, version depends on the date it was submitted: https://stackoverflow.com/help/licensing . (It's really unfortunate that they didn't pick license compatible with code; at one point they started to move to the MIT license for code, but then didn't follow through on it.)
CC BY-SA 4.0 is "compatible with code". It is, for example, GPL-compatible (see https://wiki.creativecommons.org/wiki/ShareAlike_compatibili...). It's just not designed for code.
That's a fair distinction. For the specific case of CC-by-sa 4.0, it's possible to convert to GPLv3. That doesn't help with prior versions of CC-by-sa.
So, for the specific case of material contributed to StackOverflow on or after 2018-05-02, it's possible to use it under GPLv3 (including appropriate attribution), so any project compatible with GPLv3 can copy it with attribution. Any material before that point is not safe to copy.
It does help with prior versions, since you can use CC BY-SA 3.0 material under CC BY-SA 4.0, which is GPLv3-compatible. (See https://meta.stackexchange.com/a/337742/308065.) It doesn't necessarily help with future versions.
I stand corrected, thank you! I forgot that CC-by-sa included an automatic upgrade clause, unlike many other licenses that require an explicit "or any later version" to do that.
Huge difference, and companies recognized the difference, right up until "AI" hype.
> The reality is that programmers are going to see other programmers code.
You're certainly correct. It's also true that companies are going to sue over it. There's no reason to make yourself an easy lawsuit target, if it's trivial to avoid it.
There's a reason why clean-room reverse engineering exists and it's to ensure that you can't commit copyright infringement by knowing about the code you are trying to reimplement.
How is that same thing?
That's not really how LLMs are used, unless we're planning on classing gitignores as IP
LLMs are interesting because they can combine things they learn from multiple projects into a new language that doesn't feature in any of them, and pick up details from your request.
Unless you're schizophrenic enough to insist that you never even see other code it's just not a realistic problem
Honestly I've had big arguments about this IP stuff before and unless you actually have a lawyer specifically go after something or very obviously violate the GPL it's just a tactic for people to slow people they don't like down. People find a way to invent HR departments fractally.
> If someone came to you and said "good news: I memorized the code of all the open source projects in this space, and can regurgitate it on command", you would be smart to ban them from working on code at your company.
If you find a human that did that send them my way, I'll hire them.
> There is also IP taint when using "AI". We're just pretending that there's not.
I don't think anyone who's not monetarily incentivize to pretend there are IP/Copyright issues actually thinks there are. Luckily everyone is for the most part just ignoring them and the legal system is working well and not allowing them an inch to stop progress.
> I don't think anyone who's not monetarily incentivize to pretend there are IP/Copyright issues actually thinks there are.
Why do you think that about people who disagree with you? You're responding directly to someone who's said they think there's issues, and not pretending. Do you think they're lying? Did you not read what they said?
And AFAICT a lot of other people think similarly to me.
The perverse incentives to rationalize are on the side of the people looking to exploit the confusion, not the people who are saying "wait a minute, what you're actually doing is..."
So a gold rush person claiming opponents must be pretending because of incentives... seems like the category of "every accusation is a confession".
No, I think they have a monetary incentive they have failed to disclose or a limited at best understanding of basic copyright law.
They can have a moral view that AI is "stealing" but they are claiming there is actually a legal issue at play.
Putting aside the notion that a monetary incentive is invalid for some reason, there are also plenty of open source authors who are pretty unimpressed that they only ask for attribution and even that's being laundered away via weights. If their licence isn't being respected that's certainly a legal issue.
Pretty weird motivation for writing open source software. Just saying.
I think there are ramifications to making the argument that learning is stealing.
I don't want my children to pay a license fee to their school or their textbook publishers for what they learn in school.
Learning isn't stealing, but learning can absolutely lead to stealing. (Edit: This is why bring able to demostrate that you did not learn from a copyrighted work when trying to replicate and compete with it can be an important defense.)
The amount of IP risk caused by USING (not training) AI models to produce code, especially wholesale commercial code that competes with code that was contained in the training data, is poorly understood.
There is same pretensiom with "hallucinations". If I did the same, it would bs called lying or bullshitting.
> I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of
I really appreciate this point from mitchellh. Giving thoughtful constructive feedback to help a junior developer improve is a gift. Yet it would be a waste of time if the PR submitter is just going to pass it to an AI without learning from it.
Junior developers are entering a workforce where they will never not be using AI
> Junior developers are entering a workforce where they will never not be using AI
This remark seems very US-centric to me. In my observation, many people are much more skeptical concerning whether AI is actually useful beyond some gimmicky applications.
Yes, in the same way junior pilots are entering a workforce where they will never not be using an autopilot.
I don't think using AI at all is forbidden, he just doesn't want AI to do the whole PR?
The requirement is explicitly
> If you are using *any kind of AI assistance* to contribute to Ghostty, it must be disclosed in the pull request.
This is sufficiently confusing that someone is asking if this applies to tab completion. They commit actually says
> trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases.
So if you take this literally you're going to be disclosing every yasnippet expansion that completes boilerplate.
The policy as written isn't sensible and I don't think it's entirely coming from a sensible place.
Junior developers need to learn how to code with AI because that's what coding is now. Not that he has to help them. But it does read a bit weird to toot your horn about how important it is to be helpful until it comes to helping people understand how to navigate the current environment then it's not worth your time.
Disclosing doesn't mean it will be declined, it's just a signal for reviewers.
The question is going to be: is it a USEFUL signal. I suspect not. And frankly, to be honest, as a senior developer who uses AI assistants routinely, I would consider it a serious disincentive to actually submit a PR in the first place to a repository that has such a policy. Submitting a good PR is hard work. Often an order of magnitude more work than the fix itself. If I think that a repository is going to not accept my PR just because I use a coding assistant, I'm going to be much less inclined to submit a PR at all.
> Junior developers need to learn how to code with AI because that's what coding is now.
Rather: this is what coding is now in some Silicon Valley bubbles.
They will still need to learn to recognise if the output from AI is good or not.
The rules can be finely adjusted when it actually becomes problematic, they're not trying to pass a law through Congress.
I’m loving today. HN’s front page is filled with some good sources today. No nonsense sensationalism or preaching AI doom, but more realistic experiences.
I’ve completely turned off AI assist on my personal computer and only use AI assist sparingly on my work computer. It is so bad at compound work. AI assist is great at atomic work. The rest should be handled by humans and use AI wisely. It all boils down back to human intelligence. AI is only as smart as the human handling it. That’s the bottom line.
> AI is only as smart as the human handling it.
I think I'm slowly coming around to this viewpoint too. I really just couldn't understand how so many people were having widely different experiences. AI isn't magic; how could I have expected all the people I've worked with who struggle to explain stuff to team members, who have near perfect context, to manage to get anything valuable across to an AI?
I was original pretty optimistic that AI would allow most engineers to operate at a higher level but it really seems like instead it's going to massively exacerbate the difference between an ok engineer and a great engineer. Not really sure how I feel about that yet but at-least I understand now why some people think the stuff is useless.
One of my mental models is that the notion of "effective engineer" used to mean "effective software developer" whether or not they were good at system design.
Now, an "effective engineer" can be a less battle-tested software developer, but they must be good at system design.
(And by system design, I don't just mean architecture diagrams: it's a personal culture of constantly questioning and innovating around "let's think critically to see what might go wrong when all these assumptions collide, and if one of them ends up being incorrect." Because AI will only suggest those things for cut-and-dry situations where a bug is apparent from a few files' context, and no ambitious idea is fully that cut-and-dry.)
The set of effective engineers is thus shifting - and it's not at all a valid assumption that every formerly good developer will see their productivity skyrocket.
I suspect that truly battle tested engineers will go up in value.
I don't think that it lowers the bar there, if anything the bar is far harsher.
If I'm doing normal coding I make X choices per time period, with Y impacts.
With AI X will go up and the Y / X ratio may ALSO go up, so making more decisions of higher leverage!
I've been starting to think of it like this:
Great Engineer + AI = Great Engineer++ (Where a great engineer isn't just someone who is a great coder, they also are a great communicator & collaborator, and love to learn)
Good Engineer + AI = Good Engineer
OK Engineer + AI = Mediocre Engineer
I recently watched a mid-level engineer use AI to summarize some our code, and he had it put together a big document describing all the various methods in a file, what they're used for, and so forth. It looked to me like a huge waste of time, as the code itself was already very readable (I say this as someone who recently joined the project), and the "documentation" the AI spit out wasn't that different than what you'd get just by running pydoc.
He took a couple days doing this, which was shocking to me. Such a waste of time that would have been better spent reading the code and improving any missing documentation - and most importantly asking teammates about necessary context that couldn't just be inferred from the code.
I hate to break it to you, but this guy probably wasn’t working at all. That sounds like a pretense to goof off.
Now I could believe an intern would do such a thing. I’ve seen a structural engineer intern spend four weeks creating a finite element model of a single concrete vault. he could have treated the top deck as a concrete beam used conservative assumptions about the loading and solved it with pen and paper in 30 minutes.
Well, said engineer is no longer working at my company. He wasn't exactly the best developer...
I sort of think of it in terms of self-deskilling.
If an OK engineer is still actively trying to learn, making mistakes, memorizing essentials, etc. then there is no issue.
On the other hand, if they're surrendering 100% of their judgment to AI, then they will be mediocre.
The same people who just copy-pasted stack overflow answers and didn't understand why or how things work are now using AI to create stuff that they also don't understand.
And for low-stakes one-time hobby projects, they're correct to do so!
lol. I am SO glad I don't have to go to StackExchange anymore. There is something toxically awful about using advice from a thread that starts with "Why doesn't my code work?".
Is there a difference between "OK" and "Mediocre"?
“Ok” I generally associate with being adequate but could obviously be better. “Mediocre” is just inadequate.
Some synonyms for mediocre: decent, middling, ordinary, so-so. It can mean inadequate, but it can also mean adequate.
Another common synonym for mediocre: has no place on a software development team. Not technically correct, admittedly, but that's how I read that word in an software engineering context. Adequate is not good enough.
I’m just talking about how I view/use it.
“Mediocre” is one of those words where common parlance doesn’t quite line up with the textbook definition. e.g. from the Oxford English Dictionary: “Of middling quality; neither bad nor good...”
I probably should have written it as “OK Engineer--“
Not Engineer + AI = Now an Engineer
Thats the reason for high valuation of AI companies.
"Engineer" and "someone who can produce code" are not the same thing.
Of all the things I would absolutely not trust the stock market to evaluate, "technical competence" is either near or at the top.
The people deciding how much OpenAI is worth would probably struggle to run first-time setup on an iPad.
I agree. Seems like people took my comment above as my opinion. It was supposed to be argument of Linkedin type AI hype generators.
It seemed like a normative statement, to be honest, so I misunderstood your point.
This fits my observations as well. With the exception that it’s sometimes the really sharp engineers who can do wonders themselves who aren’t really great at communication. AI really needs you to be verbose, and a lot of people just can’t.
At the moment, AI tools are particularly useful for people who feel comfortable browsing through large amounts of text, intuitively nudging the machine this way and that until arriving at a valuable outcome.
However, that way of working can be exasperating for those who prefer a more deterministic approach, and who may feel frustrated by the sheer amount of slightly incorrect stuff being generated by the machine.
As a summary, any scaling tech greatly exacerbates differences.
Nassim Taleb is the prophet of our times and he doesn't get enough credit.
I've been struggling to apply AI on any large scale at work. I was beginning to wonder if it was me.
But then my wife sort of handed me a project that previously I would have just said no to, a particular Android app for the family. I have instances of all the various Android technologies under my belt, that is, I've used GUI toolkits, I've used general purpose programming languages, I've used databases, etc, but with the possible exception of SQLite (which even that is accessed through an ORM), I don't know any of the specific technologies involved with Android now. I have never used Kotlin; I've got enough experience that I can pretty much piece it together when I'm reading it but I can't write it. Never used the Android UI toolkit, services, permissions, media APIs, ORMs, build system, etc.
I know from many previous experiences that A: I could definitely learn how to do this but B: it would be a many-week project and in the end I wouldn't really be able to leverage any of the Android knowledge I would get for much else.
So I figured this was a good chance to take this stuff for a spin in a really hard way.
I'm about eight hours in and nearly done enough for the family; I need about another 2 hours to hit that mark, maybe 4 to really polish it. Probably another 8-12 hours and I'd have it brushed up to a rough commercial product level for a simple, single-purpose app. It's really impressive.
And I'm now convinced it's not just that I'm too old a fogey to pick it up, which is, you know, a bit of a relief.
It's just that it works really well in some domains, and not so much in others. My current work project is working through decades of organically-grown cruft owned by 5 different teams, most of which don't even have a person on them that understands the cruft in question, and trying to pull it all together into one system where it belongs. I've been able to use AI here and there for some stuff that is still pretty impressive, like translating some stuff into psuedocode for my reference, and AI-powered autocomplete is definitely impressive when it correctly guesses the next 10 lines I was going to type effectively letter-for-letter. But I haven't gotten that large-scale win where I just type a tiny prompt in and see the outsized results from it.
I think that's because I'm working in a domain where the code I'm writing is already roughly the size of the prompt I'd have to give, at least in terms of the "payload" of the work I'm trying to do, because of the level of detail and maturity of the code base. There's no single sentence I can type that an AI can essentially decompress into 250 lines of code, pulling in the correct 4 new libraries, and adding it all to the build system the way that Gemini in Android Studio could decompress "I would like to store user settings with a UI to set the user's name, and then display it on the home page".
I think I recommend this approach to anyone who wants to give this approach a fair shake - try it in a language and environment you know nothing about and so aren't tempted to keep taking the wheel. The AI is almost the only tool I have in that environment, certainly the only one for writing code, so I'm forced to really exercise the AI.
> try it in a language and environment you know nothing about and so aren't tempted to keep taking the wheel.
That's a good insights. Its almost like to use AI tools effectively, one needs to stop caring about the little things you'd get caught up in if you were already familiar and proficient in a stack. Style guidelines, a certain idiomatic way to do things, naming conventions, etc.
A lot like how I've stopped organizing digital files into folders, sub folders etc (along with other content) and now I just just rely on search. Everything is a flat structure, I don't care where its stored or how it's organized as long as I can just search for it, that's what the computer is for, to keep track for me so I don't have to waste time organizing it myself.
Like wise for the code Generative AI produces. I don't need to care about the code itself. As long as its correct, not insecure, and performant, it's fine.
It's not 100% there yet, I still do have to go in and touch the code, but ideally I shouldn't have to, nor should I have to care what the actual code looks like, just the result of it. Let the computer manage that, not me. My role should be the system design and specification, not writing the code.
> Its almost like to use AI tools effectively, one needs to stop caring about the little things you'd get caught up in if you were already familiar and proficient in a stack. Style guidelines, a certain idiomatic way to do things, naming conventions, etc.
This makes me a little sad. Part of the joy of writing software is expressing yourself through caring about these little things. Stylistic coherence, adhering to consistent naming conventions, aligning code blocks, consistently applying patterns, respecting the way the language and platform's APIs work together rather than fighting it... heck, even tiny things like alphabetizing header declarations. None of these things make the finished product better/faster/more reliable, but all of these demonstrate something about the author: What he believes in. That he is willing to sand, polish and beautify the back of the cabinet that nobody is going to see. As Steve Jobs said:
If you surrender to the AI, you're no longer carrying the aesthetic and quality all the way through. You're abandoning the artistry. You're living with the barf because it works, and because it's much harder to go back and beautify it than it is to build it beautifully from the start.How do you know the code is correct, isn’t insecure, and performant, without caring about the code?
Not a solved problem yet, admittedly. Performance can be measured externally, but ideally we'll get to a point where certain criteria can be automatically tested/AI output be checked without manual intervention, or even before the AI actually puts text to editor.
What you are describing also seems to align with the idea that greenfield projects are well-suited for AI, whereas brownfield projects are considerably more challenging.
Brownfield projects are more challenging because of all the context and decisions that went into building things that are not directly defined in the code.
I suspect that well-engineered projects with plenty of test coverage and high-quality documentation will be easier to use AI on, just like they're easier for humans to comprehend. But you need to have somebody with the big picture still who can make sure that you don't just turn things into a giant mess once less disciplined people start using AI on a project.
Well said - language (text input) is actually the vehicle you have to transfer neural state to the engine. When you are working in a greenfield project or pure-vibe project, you can get away with most of that neural state being in the "default" probability mode. But in a legacy project, you need significantly more context to contrain the probability distributions a lot closer to the decisions which were made historically otherwise you quickly get into spaghetti-ville as the AI tries to drag the codebase towards its natural ruts.
Also, as soon as the code no longer fits within the context window of an LLM, one must resort to RAG-based solutions, which often leads to a significant decline in quality.
It's a matter of the tools not getting there though. If there was a summarization system that could compress down the structure and history of the system you are working on in a way that could then extract out a half-filled context window of the relevant bits of the code base and architecture for the task (in other words, generate that massive prompt for you), then you might see the same results that you get with Android apps.
The reason being that the boilerplate Android stuff is effectively given for free and not part of the context as it is so heavily represented in the training set, whereas the unique details of your work project is not. But finding a way to provide that context, or better yet fine-tune the model on your codebase, would put you in the same situation and there's no reason for it to not deliver the same results.
That it is not working for you now at your complex work projects is a limitation of tooling, not something fundamental about how AI works.
Aside: Your recommendation is right on. It clicked for me when I took a project that I had spent months of full-time work creating in C++, and rewrote it in idiomatic Go, a language I had never used and knew nothing about. It took only a weekend, and at the end of the project I had reviewed and understood every line of generated code & was now competent enough to write my own simple Go projects without AI help. I went from skeptic to convert right then and there.
I agree that the level of complexity of task it can do is likely to rise over time. I often talk about the "next generation" of AI that will actually be what we were promised LLMs would be, but LLMs architecturally are just not suited for. I think the time is coming when AIs "truly" (for some definition of truly) will understand architecture and systems in a way that LLMs don't and really can't, and will be able to do a lot more things than they can now, though when that will be is hard to guess. Could be next year, or AI could stall out now where it is now for the next 10. Nobody knows.
However, the information-theoretic limitation of expressing what you want and how anyone, AI or otherwise, could turn that into commits, is going to be quite the barrier, because that's fundamental to communication itself. I don't think the skill of "having a very, very precise and detailed understanding of the actual problem" is going anywhere any time soon.
Yes, but:
(1) The process of creating "a very, very precise and detailed understanding of the actual problem" is something AI is really good at, when partnered with a human. My use of AI tools got immensely better when I figured out that I should be prompting the AI to turn my vague short request into a detailed prompt, and then I spend a few iteration cycles fixing up before asking the agent to do it.
(2) The other problem of managing context is a search and indexing problem, which we are really, really good at and have lots of tools for, but AI is just so new that these tools haven't been adapted or seen wide use yet. If the limitation of the AI was its internal reasoning or training or something, I would be more skeptical. But the limitation seems to be managing, indexing, compressing, searching, and distilling appropriate context. Which is firmly in the domain of solvable, albeit nontrivial problems.
I don't see the information theoretic barrier you refer to. The amount of information an AI can keep in its context window far exceeds what I have easily accessible to my working memory.
The information theoretic barrier is in the information content of your prompt, not the ability of the AI to expand it.
But then I suppose I should learn from my own experiences and not try to make information theoretic arguments on HN, since it is in that most terrible state where everyone thinks they understand it because they use "bits" all the time, but in fact the average HN denizen knows less than nothing about it because even their definition of "bit" actively misleads them and that's about all they know.
I have a CS theory background. If by prompt you mean the full context provided, then there isn’t an effective limit. Claude now has 1M token context windows. You are not going to fill that with just a task specification. You could easily fill it in a large repo with the accumulated design history and total codebase. However this is also largely static, and could be used for fine-tuning. With fine tuning you’re back to a 1M token task specification for the unique variation of this prompt, and recent changes. You are not going to easily fill that.
I have vibe coded things like a personal finance forecast with interactive graphs in less than 30min of effort. I have 3 kids and would never have considered learning React to do that as a hobby project.
There are so many unusual or one off use cases that would have normally required me to spend several hours locating and reading API documentation that I now just hand off to the AI. I am a big believer in their value. I’m getting more done than ever.
I have had similar observations to you and tried to capture them here: https://www.linkedin.com/posts/alex-buie-35b488158_ai-neuros...
The gist being - language (text input) is actually the vehicle you have to transfer neural state to the engine. When you are working in a greenfield project or pure-vibe project, you can get away with most of that neural state being in the "default" probability mode. But in a legacy project, you need significantly more context to contrain the probability distributions a lot closer to the decisions which were made historically.
It’s like the difference between someone who can search the internet or codebase well bs someone who can’t
Using search engines is a skill
Today, I read the following in the concluding sentence of Frederik P. Brooks' essay “No Silver Bullets, Refired"[0]. I am quoting the entire chapter in full because it is so apt and ends with a truly positive message.
> Net on Bullets - Position Unchanged
> So we come back to fundamentals. Complexity is the business we are in, and complexity is what limits us. R. L. Glass, writing in 1988, accurately summarizes my 1995 views:
>> So what, in retrospect, have Parnas and Brooks said to us? That software development is a conceptually tough business. That magic solutions are not just around the corner. That it is time for the practitioner to examine evolutionary improvements rather than to wait—or hope—for revolutionary ones.
>> Some in the software field find this to be a discouraging picture. They are the ones who still thought breakthroughs were near at hand.
>> But some of us—those of us crusty enough to think that we are realists—see this as a breath of fresh air. At last, we can focus on something a little more viable than pie in the sky. Now, perhaps, we can get on with the incremental improvements to software productivity that are possible, rather than waiting for the breakthroughs that are not likely to ever come.[1]
[0]: Brooks, Frederick P.,Jr, The mythical man-month: essays on software engineering (1995), p. 226
[1]: Glass, R. L., "Glass"(column), System Development, (January 1988), pp. 4-5.
> AI is only as smart as the human handling it.
An interesting stance.
Plenty of posts in the style of "I wrote this cool library with AI in a day" were written by really smart devs who are known for shipping good quality library very quickly.
I'm right there with you, and having a similar experience at my day job. We are doing a bit of a "hack week" right now where we allow everyone in the org to experiment in groups with AI tools, especially those that don't regularly use them as part of their work - and we've seen mostly great applications of analytical approaches, guardrails and grounded generation.
It might just be my point of view, but I feel like there's been a sudden paradigm shift back to solid ML from the deluge of chatbot hype nonsense.
Sure. But a smart person using an AI is way smarter than a smart person not using an AI. Also keep in mind that the IQ of various AIs varies dramatically. The Google Search AI, for example, has an IQ in the 80s (and it shows); whereas capable paid AIs consistently score in the 120 IQ range. Not as smart as me, but close enough. And entirely capable of doing in seconds what would take me hours to accomplish, while applying every single one of my 120+ IQ points to the problem at hand. Im my opinion, really smart people delegate.
The way I've been thinking about it is that the human makes the key decisions and then the AI connects the dots.
What's a key decision and what's a dot to connect varies by app and by domain, but the upside is that generally most code by volume is dot connecting (and in some cases it's like 80-90% of the code), so if you draw the lines correctly, huge productivity boosts can be found with little downside.
But if you draw the lines wrong, such that AI is making key decisions, you will have a bad time. In that case, you are usually better off deleting everything it produced and starting again rather than spending time to understand and fix its mistakes.
Things that are typically key decisions:
- database table layout and indexes
- core types
- important dependencies (don't let the AI choose dependencies unless it's low consequence)
- system design—caches, queues, etc.
- infrastructure design—VPC layout, networking permissions, secrets management
- what all the UI screens are and what they contain, user flows, etc.
- color scheme, typography, visual hierarchy
- what to test and not to test (AI will overdo it with unnecessary tests and test complexity if you let it)
- code organization: directory layout, component boundaries, when to DRY
Things that are typically dot connecting:
- database access methods for crud
- API handlers
- client-side code to make API requests
- helpers that restructure data, translate between types, etc.
- deploy scripts/CI and CD
- dev environment setup
- test harness
- test implementation (vs. deciding what to test)
- UI component implementation (once client-side types and data model are in place)
- styling code
- one-off scripts for data cleanup, analytics, etc.
That's not exhaustive on either side, but you get the idea.
AI can be helpful for making the key decisions too, in terms of research, ideation, exploring alternatives, poking holes, etc., but imo the human needs to make the final choices and write the code that corresponds to these decisions either manually or with very close supervision.
Re: "What about my autocomplete?" which has shown up twice in this thread so far.
> As a small exception, trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases.
RTFA (RTFPR in this case)
I think this seems totally reasonable, the additional context provided is, I think, important to the requirement.
Some of the AI policy statements I have seen come across more as ideology statements. This is much better, saying the reasons for the requirement and offering a path forward. I'd like to see more of this and less "No droids allowed"
I’m not a big AI fan but I do see it as just another tool in your toolbox. I wouldn’t really care how someone got to the end result that is a PR.
But I also think that if a maintainer asks you to jump before submitting a PR, you politely ask, “how high?”
It does matter how and where a PR comes from, because reviewers are fallible and finite, so trust enters the equation inevitably. You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.
If trust didn't matter, there wouldn't have been a need for the Linux Kernel team to ban the University of Minnesota for attempting to intentionally smuggle bugs through the PR process as part of an unauthorized social experiment. As it stands, if you / your PRs can't be trusted, they should not even be admitted to the review process.
> "Do I trust where this came from?"
In an open source project I think you have to start with a baseline assumption of "trust nobody." Exceptions possibly if you know the contributors personally, or have built up trust over years of collaboration.
I wouldn't reject or decline to review a PR just because I don't trust the contributor.
Better to think in terms of distrust rather than trust.
Presumably if a contributor repeatedly made bad PRs that didn't do what they said, introduced bugs, scribbled pointlessly on the codebase, and when you tried to coach or clarify at best they later forgot everything you said and at worst outright gaslit and lied to you about their PRs... you would reject or decline to review their PRs, right? You'd presumably ban the outright.
Well that's exactly what commercial LLM products, with the aid of less sophisticated users, have already done to the maintainers of many large open source projects. It's not that they're not trusted-- they should be distrusted with ample cause.
So what if the above banned contributor kept getting other people to mindlessly submit their work and even proxy communication through -- evading your well earned distrust and bans? Asking people to at least disclose that they were acting on behalf of the distrusted contributor would be the least you would do, I hope? Or even asking them to disclose if and to what extent their work was a collaboration with a distrusted contributor?
If it comes with good documentation and appropriate tests, does that help?
The observation that inspired this policy is that if you used AI, it is likely you don't know if the code, the documentation or tests are good or appropriate.
What if you started with good documentation that you personally wrote, you gave that to the agent, and you verified the tests were appropriate and passed?
I'd extrapolate that the OP's view would be: you've still put in less effort, so your PR is less worthy of his attention than someone who'd done the same without using LLMs.
That's a pretty nice offer from one of the most famous and accomplished free software maintainers in the world. He's promising not to take a short-cut reviewing your PR, in exchange for you not taking a short-cut writing it in the first place.
> in exchange for you not taking a short-cut writing it in the first place.
This “short cut” language suggests that the quality of the submission is going to be objectively worse by way of its provenance.
Yet, can one reliably distinguish working and tested code generated by a person vs a machine? We’re well past passing Turing tests at this point.
LLMs can't count letters, their writing is boring, and you can trick them into talking gibberish. That is a long way off the Turing test, even if we were fooled for a couple of weeks in 2022.
IMO when people declare that LLMs "pass" at a particular skill, it's a sign that they don't have the taste or experience to judge that skill themselves. Or - when it's CEOs - they have an interest in devaluing it.
So yes if you're trying to fool an experienced open source maintainer with unrefined LLM-generated code, good luck (especially one who's said he doesn't want that).
We’re talking about code here, not prose.
Would you like to take the Pepsi challenge? Happy to put random code snippets in front of you and see whether you can accurately determine whether it was written by a human or an LLM.
I suppose it depends if AI is writing the tests an documentation.
> You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.
No you don’t. You can’t outsource trust determinations. Especially to the people you claim not to trust!
You make the judgement call by looking at the code and your known history of the contributor.
Nobody cares if contributors use an LLM or a magnetic needle to generate code. They care if bad code gets introduced or bad patches waste reviewers’ time.
You’re completely incorrect. People care a lot about where code came from. They need to be able to trust that code you’re contributing was not copied from a project under AGPLv3, if the project you’re contributing to is under a different license.
Stop trying to equate LLM-generated code with indexing-based autocomplete. They’re not the same thing at all: LLM-generated code is equivalent to code copied off Stack Overflow, which is also something you’d better not be attempting to fraudulently pass off as your own work.
I’m not equating any type of code generation. I’m saying that as a maintainer you have to evaluate any submission on the merits, not on a series of yes/no questions provided by the submitter. And your own judgement is influenced by what you know about the submitter.
And I’m saying, as a maintainer, you have to and are doing both, even if you don’t think you are.
For example, you either make your contributors attest that their changes are original or that they have the right to contribute their changes—or you assume this of them and consider it implicit in their submission.
What you (probably) don’t do is welcome contributions that the contributors do not have the right to make.
How does an "I didn't use AI" pledge provide any assurance/provenance that submitted code wasn't copied from an AGPLv3 reference?
It doesn’t, it provides an assurance (but not provenance) you didn’t use AI.
Assuring you didn’t include any AGPLv3 code in your contribution is exactly the same kind of assurance. It also doesn’t provide any provenance.
Conflating assurance with provenance is bogus because the former is about making a representation that, if false, exposes the person making it to liability. For most situations that’s sufficient that provenance isn’t needed.
Trust is absolutely a thing. Maintaining an open source project is an unreasonably demanding and thankless job, and it would be even more so if you had to treat every single PR as if it's a high likelihood supply-chain attack.
While true, we really should be treating every single piece of external code as though it's malicious.
No, we shouldn't. We live in a society, and that level of distrust is not just unrealistic, it's disastrous. This doesn't mean you should share your house keys with every drive by PR contributor, but neither should you treat every PR as if it's coming from Jia Tan.
> Nobody cares if contributors use an LLM or a magnetic needle to generate code.
That’s exactly opposite of what the author is saying. He mentions that [if the code is not good, or you are a beginner] he will help you get to finish line, but if it’s LLM code, he shouldn’t be putting effort because there’s no human on the other side.
It makes sense to me.
"but if it’s LLM code, he shouldn’t be putting effort because there’s no human on the other side"
That's the false equivalence right there
It's not a false equivalence. You can teach a beginner to become an intermediate (and later a master, if they stick to it). You can't teach an LLM to be better. Every piece of feedback you give to an LLM is like screaming into the void - it wastes your time, and doesn't change the LLM one iota.
"Every piece of feedback you give to an LLM is like screaming into the void - it wastes your time, and doesn't change the LLM one iota."
I think you just haven't gotten the hang of it yet, which is fine... the tooling is very immature and hard to get consistent results with. But this isn't a given. Some people do get good, steerable LLM coding setups.
As a maintainer, if you're dealing with a contributor who's sending in AI slop, you have no opportunity to prompt the LLM.
The PR effectively ends up being an extremely high-latency conversation with an LLM, via another human who doesn't have the full context/understanding of the problem.
You're totally dismissing this person's agency and their ability to learn. You're all but writing off their existence.
Steering via prompting isn't the same as fundamentally changing the LLM by teaching, as you can do with humans. I think OP understands this better than you.
Can't tell if you're responding in earnest or not here?
LLMs are trained to be steerable at inference time via context/prompting. Fine tuning is also possible and often used. Both count as "feedback" in my book, and my point is that both can be effective at "changing the LLM" in terms of its behavior at inference time.
And also clearly not what the OP means, who was trying to make a point that tuning the prompt to an otherwise stateless LLM inference job is nothing at all like teaching a human being. Mechanically, computationally, morally or emotionally. For example, humans aren't just tools; giving feedback to LLMs does little to further their agency.
The false equivalence I pointed at earlier was "LLM code => no human on the other side".
The person driving the LLM is a teachable human who can learn what's what's going on and learn to improve the code. It's simply not true that there's no person on the other side of the PR.
The idea that we should be comparing "teaching a human" to "teaching an LLM" is yet another instance of this false equivalence.
It's not inherently pointless to provide feedback on a PR with code written using an LLM, that feedback goes to the person using the LLM tools.
People are swallowing this b.s. marketing mystification of "LLMs as non human entities". But really they're fancy compilers that we have a lot to learn about.
The person operating the LLM is not a meaningfully teachable human when they're not disclosing that they're using an LLM.
IF they disclose what they've done, provided the prompts, etc. then other contributors can help them get better results from the tools. But the feedback is very different than the feedback you'd give a human that actually wrote the code in question, that latter feedback is unlikely to be of much value (and even less likely to persist).
Yep, true.
I've done things like share a ChatGPT account with a junior dev to steer them toward better prompts, actually, and that had some merit.
You haven't addressed the primary stated rationale from the linked content: "I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so."
> I wouldn’t really care how someone got to the end result that is a PR.
I can generate 1,000 PRs today against an open source project using AI. I think you do care, you are only thinking about the happy path where someone uses a little AI to draft a well constructed PR.
There's a lot ways AI can be used to quickly overwhelm a project maintainer.
In that case a more correct rule (and probably one that can be automatically enforced) for that issue is a max number of PRs or opened issues per account.
I think this is sane, although possibly not sufficient. Asking people to self-disclose AI usage is not going shield maintainers from a flood of undisclosed AI submissions.
> I can generate 1,000 PRs today against an open source project using AI.
Then perhaps the way you contribute, review, and accept code is fundamentally wrong and needs to change with the times.
It may be that technologies like Github PRs and other VCS patterns are literally obsolete. We've done this before throughout many cycles of technology, and these are the questions we need to ask ourselves as engineers, not stick our heads in the sand and pretend it's 2019.
I don't think throwing out the concept of code reviews and version control is the correct response to a purported rise in low-effort high-volume patches. If anything it's even more required.
Heck, let's throw out QA, too :-))
Why it's incorrect? And what would be the new way? AI to review the changes of AI?
If machines can iterate faster than humans, we'll need machines to do the reviewing; that means the testing/QA will be done perhaps by machines which will operate on a spec similar to what Amazon is doing with Kilo.
Before PR's existed we passed around code changes via email. Before containers we installed software on bare metal servers. And before search engines we used message boards. It's not unfathomable that the whole idea of how we contribute and collaborate changes as well. Actually that is likely going to be the /least/ shocking thing in the next few years if acceleration happens (i.e. The entire OS is an LLM that renders pixels, for example)
You're free to invent a better way, popularize it and become a millionaire.
It's not just about how you got there. At least in the United States according to the Copyright Office... materials produced by artificial intelligence are not eligible for copyright. So, yeah, some people want to know for licensing purposes. I don't think that's the case here, but it is yet another reason to require that kind of disclosure... since if you fail to mention that something was made by AI as part of a compound work you could end up losing copyright over the whole thing. For more details, see [2] (which is part of the larger report on Copyright and AI at [1]).
--
[1] https://www.copyright.gov/ai/
[2] https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...
> • The use of AI tools to assist rather than stand in for human creativity does not affect the availability of copyright protection for the output.
> • Copyright protects the original expression in a work created by a human author, even if the work also includes AI-generated material
> • Human authors are entitled to copyright in their works of authorship that are perceptible in AI-generated outputs, as well as the creative selection, coordination, or arrangement of material in the outputs, or creative modifications of the outputs.
Original expression, yes, however you should've kept reading:
"In the Office’s view, it is well-established that copyright can protect only material that is the product of human creativity. Most fundamentally, the term “author,” which is used in both the Constitution and the Copyright Act, excludes non-humans." "In the case of works containing AI-generated material, the Office will consider whether the AI contributions are the result of “mechanical reproduction” or instead of an author’s “own original mental conception, to which [the author] gave visible form.” 24 The answer will depend on the circumstances, particularly how the AI tool operates and how it was used to create the final work.25 This is necessarily a case-by-case inquiry." "If a work’s traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it."
The office has been quite consistent that works containing both human-made and AI-made elements will be registerable only to the extent that they contain human-made elements.
> if you fail to mention that something was made by AI as part of a compound work you could end up losing copyright over the whole thing
The source you linked says the opposite of that: "the inclusion of elements of AI-generated content in a larger human-authored work does not affect the copyrightability of the larger human-authored work as a whole"
The quote you pulled suggests that if the work is majority machine-generated, then it loses copyright protection.
That is, it suggests that even if there are elements of human-generated content in a larger machine-generated work, the combined work as a whole is not eligible for copyright protection. Printed page iii of that PDF talks a bit more about that:
This is what you get for skimming. :D
Just to be sure that I wasn't misremembering, I went through part 2 of the report and back to the original memorandum[1] that was sent out before the full report issued. I've included a few choice quotes to illustrate my point:
"These are no longer hypothetical questions, as the Office is already receiving and examining applications for registration that claim copyright in AI-generated material. For example, in 2018 the Office received an application for a visual work that the applicant described as “autonomously created by a computer algorithm running on a machine.” 7 The application was denied because, based on the applicant’s representations in the application, the examiner found that the work contained no human authorship. After a series of administrative appeals, the Office’s Review Board issued a final determination affirming that the work could not be registered because it was made “without any creative contribution from a human actor.”"
"More recently, the Office reviewed a registration for a work containing human-authored elements combined with AI-generated images. In February 2023, the Office concluded that a graphic novel comprised of human-authored text combined with images generated by the AI service Midjourney constituted a copyrightable work, but that the individual images themselves could not be protected by copyright. "
"In the Office’s view, it is well-established that copyright can protect only material that is the product of human creativity. Most fundamentally, the term “author,” which is used in both the Constitution and the Copyright Act, excludes non-humans."
"In the case of works containing AI-generated material, the Office will consider whether the AI contributions are the result of “mechanical reproduction” or instead of an author’s “own original mental conception, to which [the author] gave visible form.” The answer will depend on the circumstances, particularly how the AI tool operates and how it was used to create the final work. This is necessarily a case-by-case inquiry."
"If a work’s traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it."[1], pgs 2-4
---
On the odd chance that somehow the Copyright Office had reversed itself I then went back to part 2 of the report:
"As the Office affirmed in the Guidance, copyright protection in the United States requires human authorship. This foundational principle is based on the Copyright Clause in the Constitution and the language of the Copyright Act as interpreted by the courts. The Copyright Clause grants Congress the authority to “secur[e] for limited times to authors . . . the exclusive right to their . . . writings.” As the Supreme Court has explained, “the author [of a copyrighted work] is . . . the person who translates an idea into a fixed, tangible expression entitled to copyright protection.”
"No court has recognized copyright in material created by non-humans, and those that have spoken on this issue have rejected the possibility. "
"In most cases, however, humans will be involved in the creation process, and the work will be copyrightable to the extent that their contributions qualify as authorship." -- [2], pgs 15-16
---
TL;DR If you make something with the assistance of AI, you still have to be personally involved and contribute more than just a prompt in order to receive copyright, and then you will receive protection only over such elements of originality and authorship that you are responsible for, not those elements which the AI is responsible for.
--- [1] https://copyright.gov/ai/ai_policy_guidance.pdf [2] https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...
You should care. If someone submits a huge PR, you’re going to waste time asking questions and comprehending their intentions if the answer is that they don’t know either. If you know it’s generated and they haven’t reviewed it themselves, you can decide to shove it back into an LLM for next steps rather than expect the contributor to be able to do anything with your review feedback.
Unreviewed generated PRs can still be helpful starting points for further LLM work if they achieve desired results. But close reading with consideration of authorial intent, giving detailed comments, and asking questions from someone who didn't write or read the code is a waste of your time.
That's why we need to know if a contribution was generated or not.
You are absolutely right. AI is just a tool to DDoS maintainers.
Any contributor who was shown to post provably untested patches used to lose credibility. And now we're talking about accommodating people who don't even understand how the patch is supposed to work?
That’s not what I said though. LLM output, even unreviewed and without understanding, can be a useful artifact. I do it all the time - generate code, try running it, and then if I see it works well, I can decide to review it and follow up with necessary refactoring before integrating it. Parts of that can be contributed too. We’re just learning new etiquettes for doing that productively, and that does includes testing the PR btw (even if the code itself is not understood or reviewed).
Example where this kind of contribution was accepted and valuable, inside this ghostty project https://x.com/mitchellh/status/1957930725996654718
If the AI slop was that valuable a project regular, who actually knows and understands the project, would be just as capable of asking the AI to produce it.
Not according to ghostty maintainer Hashimoto per above.
It takes attempts, verifying the result behaves as desired, and iterative prompting to adjust. And it takes a lot of time to wait on agents in between those steps (this work isn’t a one shot response). You’re being reductive.
We may be talking cross purposes. I read the grandparent poster discussing provably untested patches.
I have no clue in ghostty but I've seen plenty of stuff that doesn't compile much less pass tests. And I assert there is nothing but negative value in such "contributions".
If real effort went into it, then maybe there is value-- though it's not clear to me: When a project regular does the same work then at least they know the process. Like if there is some big PR moving things around at least the author knows that it's unlikely to slip in a backdoor. Once the change is reduced to some huge diff, it's much harder to gain this confidence.
In some projects direct PRs for programmatic mass renames and such have been prohibited in favor of requiring submission of the script that produces the change, because its easier to review the script carefully. The same may be necessary for AI.
This whole original HN post is about ghostty btw
Having the original prompts (in sequence and across potentially multiple models) can be valuable but is not necessarily useful in replicating the results because of the slot machine nature of it
> This whole original HN post is about ghostty btw
Sure though I believe few commenters care much about ghostty specifically and are primarily discussing the policy abstractly!
> because of the slot machine nature of it
One could use deterministically sampled LLMs with exact integer arithmetic... There is nothing fundamental preventing it from being completely reproducible.
Can't do that with state of the art LLMs and no sign of that changing (as they like to retain control over model behaviors). I would not want to use or contribute to a project that embraces LLMs yet disallows leading models.
Besides, the output of an LLM is not really any more trustworthy (even if reproducible) than the contribution of an anonymous actor. Both require review of outputs. Reproducibility of output from prompt doesn't mean that the output followed a traceable logic such that you can skip a full manual code review as with your mass renaming example. LLMs produce antagonistic output from innocuous prompting from time to time, too.
> is that they don’t know either
It would be nice if they did, in fact, say they didn't know. But more often they just waste your time making their chatbot argue with you. And the chatbots are outrageous gaslighters.
All big OSS projects have had the occasional bullshitter/gaslighter show up. But LLMs have increased the incidence level of these sorts of contributors by many orders of magnitude-- I consider it an open question if open-public-contribution opensource is viable in the world post LLM.
There was some post that comes to mind of an example of this. Some project had a security issue reported that was not a security issue, and when asking questions it became extremely obvious that someone was just feeding the conversation into an LLM. There was no security issue. I can imagine this is happening more and more as people are trying to slam in LLM generated code everywhere.
Everyone promoting LLMs, especially on HN, claim that they're expertly using them by using artisanal prompts and carefully examining the output but.. I'm honestly skeptical. Sure, some people are doing that (I do it from time to time). But I've seen enough slop to think that more people are throwing around code that they barely understand than these advocates care to admit .
Those same people will swear that they did due diligence, but why would they admit otherwise? And do they even know what proper due diligence is? And would they still be getting their mythical 30%-50% productivity boost if they were actually doing what they claimed they were doing?
And that is a problem. I cannot have a productive code review with someone that does not even understand what their code is actually doing, much less trade offs that were made in an implementation (because they did not consider any trade offs at all and just took what the LLM produced). If they can't have a conversation about the code at all because they didn't bother to read or understand anything about it, then theres nothing I can do except close the PR and tell them to actually do the work this time.
The ghostty creator disagrees re: the productivity of un-reviewed generated PRs: https://x.com/mitchellh/status/1957930725996654718
Agreed. As someone who uses AI (completion and Claude Code), I'll disclose whenever asked. But I disagree that it's "common courtesy" when not explicitly asked; since many people (including myself) don't mind and probably assume some AI, and it adds distraction (another useless small indicator; vaguely like dependabot, in that it steals my attention but ultimately I don't care).
It’s not just common courtesy to disclose, it’s outright fraud not to disclose.
That's nonsense. It's like feeling you need to disclose that your IDE has autocomplete. Nobody discloses that, since it's ridiculous. You only disclose that you used Claude Code if you are not certain of the result (e.g. you think it is correct, but the maintainer might be a better judge).
If it's exactly the same as what you'd have written manually, and you are confident it works, then what's the point of disclosure?
It’s completely different from an IDE’s autocomplete because autocomplete in an IDE is only helping you type identifiers that already exist in your codebase or in any SDKs you’re using.
An LLM is regurgitating things from outside that space, where you have no idea of the provenance of what it’s putting into your code.
It doesn’t just matter that the code you’re contributing to a project is correct, it matters quite a lot if it’s actually something you’re allowed to contribute.
- You can’t contribute code that your employer owns to a project if they don’t want you to. - You can’t contribute code under a license that the project doesn’t want you to use. - And you can’t contribute code written by someone else and claim it’s your intellectual property without some sort of contract in place to grant that.
If you use an LLM to generate code that you’re contributing, you have both of the latter two problems. And all of those apply *even if* the code you’re contributing is identical to what you’d have written by hand off the top of your head.
When you contribute to a project, you’re not just sending that project a set of bits, you’re making attestations about how those bits were created.
Why does this seem so difficult for some supposed tech professionals to understand? The entire industry is intellectual property, and this is basic “IP 101” stuff.
> Why does this seem so difficult for some supposed tech professionals to understand?
Maybe because 99% of people that complain about this complain about problems that never occur in 99% of the cases they cite. My employer isn’t going to give a shit that code that I’ve written for their internal CRUD app gets more or less directly copied into my own. There’s only one way to do that, it was already in my head before I wrote it for them, and it’ll still be in after. As long as I’m not directly competing with their interests, what the hell do they care.
> When you contribute to a project, you’re not just sending that project a set of bits, you’re making attestations about how those bits were created.
You are really not. You are only doing that if the project requires some attestation of provenance. I can tell you that none of mine do.
It should be. You didn’t write generated code, why should I spend my life reading it?
If you want me to put in the effort- you have to put it in first.
Especially considering in 99% of cases even the one who generated it didn’t fully read/understand it.
No one is forcing you to read it. Feel free to have your own AI judge if you should merge it or even just YOLO merge it. The end goal of people trying to get code merged is not to have you read it. It's to improve the software. Whether code improves the software or not is orthogonal to if the code was written by hand.
The reason it's common courtesy is out of respect for the reviewer/maintainer's time. You need to let em know to look for the kind of idiotic mistakes LLMs shit out on a routine basis. It's not a "distraction", it's extremely relevant information. On the maintainer's discretion, they may not want to waste their time reviewing it at all, and politely or impolitely ask the contributor to do it again, and use their own brain this time. It also informs them on how seriously to take this contributor in the future, if the work doesn't hold water, or indeed, even if it does, since the next time the contributor runs the LLM lottery the result may be utter bullshit.
Whether it's prose or code, when informed something is entirely or partially AI generated, it completely changes the way I read it. I have to question every part of it now, no matter how intuitive or "no one could get this wrong"ish it might seem. And when I do, I usually find a multitude of minor or major problems. Doesn't matter how "state of the art" the LLM that shat it out was. They're still there. The only thing that ever changed in my experience is that problems become trickier to spot. Because these things are bullshit generators. All they're getting better at is disguising the bullshit.
I'm sure I'll gets lots of responses trying to nitpick my comment apart. "You're holding it wrong", bla bla bla. I really don't care anymore. Don't waste your time. I won't engage with any of it.
I used to think it was undeserved that we programmers called ourselved "engineers" and "architects" even before LLMs. At this point, it's completely farcical.
"Gee, why would I volunteer that my work came from a bullshit generator? How is that relevant to anything?" What a world.
FWIW, I can say from direct experience people that other people are watching and noting when people are submitting AI slop as their own work, and taking note to never hire these people. Beyond the general professional ethics, it makes you harder to distinguish from malicious parties and other incompetent people LARPing as having knowledge that they don't.
So fail to disclose at your own peril.
If you don't disclose the use of
- books
- search engines
- stack overflow
- talking to a coworker
then it's not clear why you would have to disclose talking to an AI.
Generally speaking, when someone uses the word "slop" when talking about AI it's a signal to me that they've been sucked into a culture war and to discount what they say about AI.
It's of course the maintainer's right to take part in a culture war, but it's a useful way to filter out who's paying attention vs who's playing for a team. Like when you meet someone at a party and they bring up some politician you've barely heard of but who their team has vilified.
> then it's not clear why you would have to disclose talking to an AI.
It’s explained right there in the PR:
> The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
That is not true of books, search engines, stack overflow, or talking to a worker, because in all those cases you still had to do the work yourself of comprehending, preparing, and submitting the patch. This is also why they ask for a disclosure of “the extent to which AI assistance was used”. What about that isn’t clear to you?
You should add citations to books, stack overflow posts and colleagues you consult with, yes.
The reality is as someone that helps maintain several OSS projects you vastly underestimate the problem that AI-assisted tooling has created.
On the one hand, it's lowered the barrier to entry for certain types of contributions. But on the other hand getting a vibe-coded 1k LOC diff from someone that has absolutely no idea how the project even works is a serious problem because the iteration cycle of getting feedback + correctly implementing it is far worse in this case.
Also, the types of errors introduced tend to be quite different between humans and AI tools.
It's a small ask but a useful one to disclose how AI was used.
When one side has much more "scalability" than the other, then the other side has very strong motivation to match up.
- People use AI to write cover letters. If the companies don't filter out them automatically, they're screwed.
- Companies use AI to interview candidates. No one wants to spend their personal time talking to a robot. So the candidates start using AI to take interviews for them.
etc.
If you don't at least tell yourself that you don't allow AI PRs (even just as a white lie) you'll one day use AI to review PRs.
Both sides will use AI and it will ultimately increase economic productivity.
Imagine living before the invention of the printing press, and then lamenting that we should ban them because it makes it "too easy" to distribute information and will enable "low quality" publications to have more reach. Actually, this exact thing happened, but the end result was it massively disrupted the world and economy in extremely positive ways.
> Both sides will use AI and it will ultimately increase economic productivity.
Citation needed, I don’t think the printing press and gpt are in any way comparable.
GPT and compilers are though.
Compilers don’t randomly fail to compile code that is too difficult for them to understand. Llvm makes sure that I never have to learn assembly, gpt doesn’t guarantee at all that I don’t have to learn to code.
The mental gymnastics the parent poster went through to equate an LLM to the printing press in this sense are mind-boggling.
Ironically, I thought your parent commenter had to go through mental gymnastics to say that their parents analogy of the printing press isn't applicable to an LLM. Neither you nor your parent gave me any satisfactory reasons why they aren't similar, just your mental superiority as proof that oceanplexian must be wrong.
> Imagine living before the invention of the printing press, and then lamenting that we should ban them because it makes it "too easy" to distribute information
Imagine seeing “rm -rf / is a function that returns “Hello World!” and thinking “this is the same thing as the printing press”
https://bsky.app/profile/lookitup.baby/post/3lu2bpbupqc2f
> Both sides will use AI and it will ultimately increase economic productivity.
In some cases sure but it can also create the situation where people just waste time for nothing (think AI interviewing other AIs - this might generate GDP by people purchasing those services but I think we can all agree that this scenario is just wasting time and resource without improving society).
Imagine being so deluded and disconnected that you actually believe that AI has any similarity with the printing press regarding the benefits to The People.
Whether the output of AI can be copyrighted remains a legal minefield, so if I were running a project where copyright-based protections are important (say, anything GPL) I would want to know if a PR contained them.
If a maintainer asks me to jump through too many stupid hoops, I'll just not contribute to the software.
That said, requiring adequate disclosure of AI is just fair. It also suggests that the other side is willing to accept AI-supported contributions (without being willing to review endless AI slop that they could have generated themselves if they had the time to read it).
I would expect such a maintainer to respond fairly to "I first vibecoded it. I then made manual changes, vibecoded a test, cursorily reviewed the code, checked that the tests provide good coverage, ran both existing and new tests, and manually tested the code."
That fair response might be a thorough review, or a request that I do the thorough review before they put in the time, but I'd expect it to be more than a blatant "nope, AI touched this, go away".
You have other choices, such as not contributing.
I won't put it as "just another tool". AI introduces a new kind of tool where the ownership of the resulting code is not straightforward.
If, in the dystopian future, a justice court you're subjected to decides that Claude was trained on Oracle's code, and all Claude users are possibly in breach of copyright, it's easier to nuke from orbit all disclosed AI contributions.
> if a maintainer asks you to jump before submitting a PR, you politely ask, “how high?”
or say "fork you."
As a project maintainer, you shouldn't make rules unenforceable rules that you and everyone else know people will flout. Doing so comes makes you seem impotent and diminishes the respect people have for rules in general.
You might argue that by making rules, even futile ones, you at least establish expectations and take a moral stance. Well, you can make a statement without dressing it up as a rule. But you don't get to be sanctimonious that way I guess.
Except you can enforce this rule some of the time. People discover that AI was used or suspect it all the time, and people admit to it after some pressure all the time.
Not every time, but sometimes. The threat of being caught isn't meaningless. You can decide not to play in someone else's walled garden if you want but the least you can do is respect their rules, bare minimum of human decency.
Except the other way happens too.
You get someone that didn't use AI getting accused of using AI and eventually telling people to screw off and contributing nothing.
If their work was difficult to distinguish from AI then that sounds like a win too.
It. doesn't. matter.
The only legitimate reason to make a rule is to produce some outcome. If your rule does not result in that outcome, of what use is the rule?
Will this rule result in people disclosing "AI" (whatever that means) contributions? Will it mitigate some kind of risk to the project? Will it lighten maintainer load?
No. It can't. People are going to use the tools anyway. You can't tell. You can't stop them. The only outcome you'll get out of a rule like this is making people incrementally less honest.
You’re basically saying “if a rule can be broken, it will be, therefore rules are useless.”
If someone really wants to commit fraud they’re going to commit fraud. (For example, by not disclosing AI use when a repository requires it.) But if their fraud is discovered, they can still be punished for it, and mitigating actions taken. That’s not nothing, and does actually do a lot to prevent people from engaging in such fraud in the first place.
There are plenty of argumentative and opinionated reasons to say it matters, but there is one that can't really be denied - reviewers (and project maintainers, even if they aren't reviewers) are people whose time deserves to be respected.
If this rule discourages low quality PRs or allows reviewers to save time by prioritizing some non-AI-generated PRs, then it certainly seems useful in my opinion.
> Will it lighten maintainer load?
Yes that is the stated purpose, did you read the linked GitHub comment? The author lays out their points pretty well, you sound unreasonably upset about this. Are you submitting a lot of AI slop PRs or something?
P.S Talking. Like. This. Is. Really. Ineffective. It. Makes. Me. Just. Want. To. Disregard. Your. Point. Out. Of. Hand.
The utility of the rule is so that you can cheaply nuke non-conforming contributors from orbit when you detect their undisclosed AI use. Vs having to deal with the flood of low quality contributions on a individually reviewed basis.
Sometimes you can tell.
Unenforceable rules are bad, but if you tweak the rule to always require some sort of authorship statement (e.g. "I wrote this by hand" or "I wrote this with Claude"), then the honor system will mostly achieve the desired goal of calibrating code review effort.
> As a project maintainer, you shouldn't make rules unenforceable rules
Total bullshit. It's totally fine to declare intent.
You are already incapable of verifying / enforcing that a contributor is legally permitted to submit a piece of code as their own creation (Signed-off-by), and do so under the project's license. You won't embark on looking for prior art, for the "actual origin" of the code, whatever. You just make them promise, and then take their word for it.
And if they’re discovered to not be keeping their word, there can be consequences imposed and mitigating actions taken. Rules can’t prevent bad actions 100% of the time, but they can substantially increase the risk of bad actions.
We keep talking about “AI replacing coders,” but the real shift might be that coding itself stops looking like coding. If prompts become the de facto way to create applications/developing systems in the future, maybe programming languages will just be baggage we’ll need to unlearn.
Programming languages were a nice abstraction to accommodate our inability to comprehend complexity - current day LLMs do not have the same limitations as us.
The uncomfortable part will be what happens to PRs and other human-in-the-loop checks. It’s worthwhile to consider that not too far into the future, we might not be debugging code anymore - we’ll be debugging the AI itself. That’s a whole different problem space that will need an entirely new class of solutions and tools.
This fundamentally misunderstands why programming languages exist. They're not required because "we can't understand complexity". They were invented because we need a way to be very specific about what we want the machine to do. Whether it's the actual physical hardware we're talking to when writing assembly, or it's an abstract machine that will be translated to the hardware like in C or Java, the key point is that we want to be specific.
Natural language can be specific, but it requires far too many words. `map (+ 1) xs` is far shorter to write than "return a list of elements by applying a function that adds one to its argument to each element of xs and collecting the results in a separate list", or similar.
Fair enough - I’m just the messenger though, observing the current trends and extrapolating from there. Let’s talk about “AGENTS.md” files quickly. We’re specifying what I consider “rules” in plain language. Even lint rules (instead of creating a lint config). Could be a matter of convenience, and if it gets us 80% of the way, why not?
I believe it won’t be long before we have exceptional “programmers” who have mastered the art of vibe coding. If that does become the de facto standard for 80% programming done, then it’s not a long stretch from there that we might skip programming languages altogether. I’m simply suggesting that if you’re not going to examine the code, perhaps someone will eliminate that additional layer or step altogether, and we might be pleasantly surprised by the final result.
All we need to do is prompt an LLM with such specificity that it does exactly what we want the machine to do.
Good idea! We can have some sort of standard grammar that we use to prompt the LLM such that it deterministically gives us the result we ask for. We then constrain all prompts to match that grammar. Some sort of language describing programs.
How does this not lead to a situation where no honest person can use any AI in their submissions? Surely pull requests that acknowledge AI tooling will be given significantly less attention, on the grounds that no one wants to read work that they know is written by AI.
I don't think this is the case. Mitchell writes that he himself uses LLMs, so it's not black and white. A PR author who has a deep understanding of their changes and used an LLM for convenience will be able to convey this without losing credibility imo
Make a knowledgeable reply and mention you used chat-gpt - comment immediately buried.
Make a knowledgeable reply and give no reference to the AI you used- comment is celebrated.
We are already barreling full speed down the "hide your AI use" path.
I doubt a PR is going to be buried if it's useful, well designed, good code, etc, just because of this disclosure. Articulate how you used AI and I think you've met the author's intent.
If the PR has issues and requires more than superficial re-work to be acceptable, the authors don't want to spend time debugging code spit out by an AI tool. They're more willing to spend a cycle or two if the benefit is you learning (either generally as a dev or becoming more familiar with the project). If you can make clear that you created or understand the code end to end, then they're more likely to be willing to take these extra steps.
Seems pretty straightforward to me and thoughtful by the maintainers here.
> I doubt a PR is going to be buried if it's useful, well designed, good code, etc, just because of this disclosure
If that were the case, why would this rule be necessary, if it indeed is the substance that matters? AI generated anything has a heavy slop stigma right now, even if the content is solid.
This would make for an interesting experiment to submit a PR that was absolute gold but with the disclaimer it was generated with help of ChatGPT. I would almost guarantee it would be received with skepticism and dismissals.
The rule is necessary because the maintainers want to build good will with contributors and if a contributor makes a bad PR but could still learn from it then they will put effort into it. It's a "if you made a good effort, we'll give you a good effort" and using AI tools gives you a very low floor for what "effort" is.
If you make a PR where you just used AI, it seems to work, but didn't go further then the maintainers can go "well I had a look, it looks bad, you didn't put effort in, I'm not going to coach you through this". But if you make a PR where you go "I used AI to learn about X then tried to implement X myself with AI writing some of it" then the maintainers can go "well this PR doesn't look good quality but looks like you tried, we can give some good feedback but still reject it".
In a world without AI, if they were getting a lot of PRs from people who obviously didn't spend any time on their PRs then maybe they would have a "tell us how long this change took you" disclosure as well.
The author explains why
> While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
If it's bad code from a person he'll help them get it fixed. If it's bad code from an AI why bother?
The last three GitHub issues I ran across when looking something up had people literally copy pasting the entire ChatGPT response as their comment. It feels like I'm living in some crazy dystopia when several _different_ people post a 30+ line message that's 95% the same. I'm sorry but I refuse to interact with people who do this, if I wanted to talk to a computer I'd do it myself.
HN works that way but Mitchell said he isn't opposed to AI. You have to know the vibe of your environment.
It just might. But if people generate a bias against AI generated code because AI can generate massive amounts of vaguely correct looking yet ultimately bad code then that seems like an AI problem not a people problem. Get better, AI coding tools.
No one is saying to not use AI. The intent here is to be honest about AI usage in your PRs.
You ask this as if it’s a problem.
i'm happy to read work written by AI and it is often better than a non-assisted PR
Isn't that a good thing?
What, building systems where we’re specifically incentivised not to disclose ai use?
Submitting a PR also means you’re not submitting code copied from elsewhere without calling that out and ensuring license compatibility, we don’t refer to that as incentivizing lying about the origin of submitted code.
Fraud and misrepresentation are always options for contributors, at some point one needs to trust that they’re adhering to the rules that they agreed to adhere to.
It might encourage people to be dishonest, or to not contribute at all. Maybe that's fine for now, but what if the next generation come to rely on these tools?
Good point. That's the point exactly. Don't use AI for writing your patch. At all.
Why are you surprised? Do companies want to hire "honest" people whose CVs were written by some LLM?
> Do companies want to hire "honest" people whose CVs were written by some LLM?
Yes, some companies do want to hire such people, the justification given is something along the lines of "we need devs who are using the latest tools/up to date on the latest trends! They will help bring in those techniques and make all of our current devs more productive!". This isn't a bad set of motivations or assumptions IMO.
Setting aside what companies _want_, they almost certainly are already hiring devs with llm-edited CVs, whether they want it or not. Such CVs/resumes are more likely to make it through HR filters.
I don't know if future generations will agree with this sentiment, in which case we lock ourselves out of future talent (i.e. those that use AI to assist, not to completely generate). The same arguments were made about Photoshop once upon a time.
> Do companies want to hire "honest" people whose CVs were written by some LLM?
Unfortunately yes, they very much seem to. Since many are using LLMs to assess CVs, those which use LLMs to help write their CV have a measured advantage.
This is such an important conversation. At Delve, we use AI extensively to automate compliance workflows, but we're always transparent about it with our customers. The real question isn't whether AI was used, but whether it adds genuine value. Does the AI actually solve a problem? Is it making people's lives easier? Can you explain how it works? The best AI tools feel invisible - they just make complex tasks simple. The worst ones make you constantly aware you're dealing with AI because they're clunky or unreliable. Transparency builds trust. If your AI tooling is genuinely helpful, you should be proud to talk about it.
> Or a more detailed disclosure:
> I consulted ChatGPT to understand the codebase but the solution was fully authored manually by myself.
What's the reasoning for needing to disclose this?
Helps the maintainer focus their efforts during review. If you wrote that exact sentence, a maintainer would keep "understanding of the codebase" as a place for potential sources of incorrect assumptions higher in mind than otherwise.
In my experience, if there's one thing the AI assistants do really well, it's understanding existing code and summarizing it or pinpointing specific things I'm looking for.
I'm not really a developer but I routinely need to go dig through codebases looking for answers and AI assistants have made this so much faster. This has been a boon for me.
I like the pattern of including each prompt used to make a given PR, yes, I know that LLM's aren't deterministic, but it also gives context of the steps required to get to the end state.
I’m using specsytory in vscode + cursor for this - it keeps a nice little md doc of all your LLM interactions, and you can check that into source control if you like so it’s included in pull requests, and can be referenced during code review.
I've been doing that for most of the commits in this project as an experiment, gemini, human, or both. Not sure what I'm going to do with that history, but I did at least want to start capturing it
https://github.com/blebbit/at-mirror/commits/main/
It's ridiculous and impractical, honestly. A single AI-generated PR would likely involve at least 10-20 prompts, interspersed with testing, manual edits to context / guideline files without which those prompts don't have the same effect, manual coding, and more. A screen recording would do better.
Is there really no logging capability with these tools that would track all of that prompting/testing/editing/inputting?
Sure, screen recorders exist.
And if contributions are that unwelcome, then it's better not to contribute. There has to be some baseline level of trust that the contributor is trying to do the right thing; I get enough spying from the corporations in my life.
I'm not talking screen recorder but a log file where I could be given it and use it for input and repeat the work exactly as it was done. Sort of like how one could do the same with their bash history file to repeat an initially exploratory analysis effort using the same exact commands. I'm surprised that isn't already a capability given the business interest with AI. One would think they would like to cache these prompt flows to ensure long term fidelity in process.
Does that exist for VSCode?
If not, why would it exist for VSCode + a variety of CLI tools + AI? Anyhow, saving the exact prompt isn't super useful; the response is stochastic.
VScode has a history capability. You can't save a prompt + a seed to recover the response? That seems like it would be highly useful.
Doesn't work. People will include fake prompts, the real ones are way too personal. You can learn a lot from how people use these tools.
That's like saying "sorry, source code is too personal. In my 'open' project you get only binaries".
... and then I think about all the weights only "open" AI projects and walk off in disgust.
Keep in mind that in industries where people code but aren't really programmers this literally does happen, sometimes very "big" people will be basically scared to share their code because it won't be very good.
But anyway what I mean is that code is us speaking like a computer, LLMs are the other way around, you can see a lot from how someone interacts with the machine.
not just non-programmers. It's a common problem with jr programmers and good programmer internships (for example) make it a point of forcing the interns to expose themselves ASAP to get over it.
I think if everyone goes into it knowing that it'll be part of what they publish it would be less of an issue.
I mean, unless you're all a bunch of freaks who have instructed your LLM to cosplay as Slave Leia and can't work otherwise, in which case your issues are beyond my pay grade. :P
I still do not understand how one can integrate "AI" code into a project with a license at all. "AI" code is not copyrightable, "AI" cannot sign a contributor agreement.
So if the code is integrated, the license of the project lies about parts of the code.
“AI” can’t, but the person running the tool can.
The contributor is the human that chose to run the LLM, not the “AI” itself - so the real question is, why isn’t the human’s code copyrightable, and why can’t the human sign a contributor agreement?
Besides, this stuff is not what the author is concerned about:
> I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code … I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort.
They want to coach aspiring contributors based on code they’ve written themselves, not based on how they prompt their AI.
It’s a matter of how they enjoy spending their time.
> I still do not understand
Your question makes sense. See U.S. Copyright Office publication:
> If a work's traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it.
> For example, when an AI technology receives solely a prompt from a human and produces complex written, visual, or musical works in response, the “traditional elements of authorship” are determined and executed by the technology—not the human user...
> For example, if a user instructs a text-generating technology to “write a poem about copyright law in the style of William Shakespeare,” she can expect the system to generate text that is recognizable as a poem, mentions copyright, and resembles Shakespeare's style. But the technology will decide the rhyming pattern, the words in each line, and the structure of the text.
> When an AI technology determines the expressive elements of its output, the generated material is not the product of human authorship. As a result, that material is not protected by copyright and must be disclaimed in a registration application.
> In other cases, however, a work containing AI-generated material will also contain sufficient human authorship to support a copyright claim. For example, a human may select or arrange AI-generated material in a sufficiently creative way that “the resulting work as a whole constitutes an original work of authorship.”
> Or an artist may modify material originally generated by AI technology to such a degree that the modifications meet the standard for copyright protection. In these cases, copyright will only protect the human-authored aspects of the work, which are “independent of” and do “not affect” the copyright status of the AI-generated material itself.
> This policy does not mean that technological tools cannot be part of the creative process. Authors have long used such tools to create their works or to recast, transform, or adapt their expressive authorship. For example, a visual artist who uses Adobe Photoshop to edit an image remains the author of the modified image, and a musical artist may use effects such as guitar pedals when creating a sound recording. In each case, what matters is the extent to which the human had creative control over the work's expression and “actually formed” the traditional elements of authorship.
> https://www.federalregister.gov/documents/2023/03/16/2023-05...
In any but a pathological case, a real contribution code to a real project has sufficient human authorship to be copyrightable.
> the license of the project lies about parts of the code
That was a concern pre-AI too! E.g. copy-past from StackOverflow. Projects require contributors to sign CLAs, which doesn't guarantee compliance, but strengthens the legal position. Usually something like:
"You represent that your contribution is either your original creation or you have sufficient rights to submit it."
I don’t see much benefit from the disclosure alone. Ultimately, this is code that needs to be reviewed. There is going to continue to be more and more AI assisted code generation, to the point where we see the same level of adoption of these tools as "Autocomplete". Why not solve this through tooling? I have had great effect with tools like Greptile, Cursor's BugBot and Claude Code.
If the code is obviously low quality and AI-generated then it doesn't need to be fully reviewed actually. You can just reject the PR.
> You can just reject the PR.
And with a better and more useful response. Instead of wasting time on the technical details, you can give feedback like "this isn't the sort of change that AI is likely to be helpful with, though if you want to keep trying make at least sure your PRs pass the tests." or "If you'd like to share your prompts we might be able to make some suggestions, we've found on this project it's useful to include <X>".
Sure it needs to be reviewed. But the author does more than just reviewing, they help the person submitting the PR to improve their PR. If the other side is an AI, it can save them some time.
Do I also have to disclose using tab completion? My IDE uses machine learning for completion suggestions.
Do I need to disclose that I wrote a script to generate some annoying boilerplate? Or that my IDE automatically templates for loops?
It's a spectrum, isn't it? I wouldn't want to waste my time reviewing a bunch of repetitive code generated from some script or do something like review every generated template instantiation in a C++ code base. I would want to review the script/template definition/etc., but what's the equivalent for AI? Should the review just be the prompt(s)?
Edit: Also, it's always good to provide maximal context to reviewers. For example, when I use code from StackOverflow I link the relevant answer in a comment so the reviewer doesn't have to re-tread the same ground I covered looking for that solution. It also gives reviewers some clues about my understanding of the problem. How is AI different in this regard?
No, it explicitly says that you don't need to disclose tab completion.
If you're not sure, it's probably safer to just mention it.
> Do I also have to disclose using tab completion? My IDE uses machine learning for completion suggestions.
Yes, you have to disclose it.
> Do I need to disclose that I wrote a script to generate some annoying boilerplate?
You absolutely need to disclose it.
> Or that my IDE automatically templates for loops?
That's probably worth disclosing too.
I always appreciated Claude Code's commit authoring. Whereas I think a lot of people were offended that "their" work was being overshadowed by an AI's signature.
We probably will do similar in the Bun repo. We use AI tools a lot and don’t discourage it, but we also label when a PR was generated by claude both via github label and also by leaving in the “Generated with claude code” textual watermark.
In my personal projects I also require all contributors to disclose rather they’ve used an editor with any autocomplete features enabled.
Heh, that's a great way to make a point, but right now AI is nowhere near what a traditional editor autocomplete is. Yes you can use it that way, but it's by no means limited to that. If you think of AI as a fancy autocomplete, that's a good personal philosophy, but there are plenty of people that aren't using it that way
The line isn’t as clear as you might think, eg jetbrains has a mini on-device neural net powered autocomplete:
https://www.jetbrains.com/help/idea/full-line-code-completio...
Notably, tab completion is an explicltly called-out exception to this policy, as detailed in the changed docs.
Autocomplete is, for the most part, a syntactic tool. AI attempts to guide the semantics of the code generated
This seems misguided. The human submitter is accountable for the code submitted. Doesn’t matter if the AI built it, a contractor in another timezone, or ten thousand monkeys typing away.
Github provides a profile but it’s not meant to make this kind of assessment. Basically a trust badge/metrics of some sort you could check before engaging with someone’s PR.
One way is to just name the LLM as the author for commits it makes. You can be the committer so long as you just accurately name the author.
I see two things here.
1. The world has fundamentally changed due to LLMs. You don't know where a code submission falls between "written thoroughly with eternal vigilance" vs "completely vibe-coded" since it's now trivially to generate the later. There's no going back. And a lot of comments here seem stuck on this point.
2. The maintainer naively or stubbornly imagines that he can get everyone to pre-sort their code between the two buckets through self-reporting.
But that's futile.
It's like asking someone if they're a good person on a date because you don't want to waste your time with bad people. Unfortunately, that shortcut doesn't exist.
Now, maybe going forward we will be forced to come up with real solutions to the general problem of vetting people. But TFA feels like more of a stunt than a serious pitch.
> maybe going forward we will be forced to come up with real solutions to the general problem of vetting people
The monekeys paw closes a finger and now you need a formal certification, professional license, and liability insurance to publish software or source code.
In a nondismissive way, I see things like this (the gh issue) as part of the reactionary movement / counter culture of our time.
People want to feel agency and will react to mainstream pressures. And make up whatever excuses along the way to justify what theyre feeling.
It's not about "feeling agency" or fabricating a justification. As the PR says:
"AI tooling must be disclosed for contributions
I think, at this stage of AI, it is a common courtesy to disclose this.
In a perfect world, AI assistance would produce equal or higher quality work than any human. That isn't the world we live in today, and in many cases it's generating slop. I say this despite being a fan of and using them successfully myself (with heavy supervision)! I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was.
The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code."
I don't think it has anything to do with being a reactionary movement or counter culture. If it were I would expect, among other things, that it would absolutely prohibit the use of AI entirely rather than just require disclosure.
The background is that many higher profile open source projects are getting deluged by low quality AI slop "contributions", not just crappy code but when you ask questions about it you sometimes get an argumentative chatbot lying to you about what the PR does.
And this latest turn has happened on top of other trends in 'social' open source development that already had many developers considering adopting far less inclusive practice. RETURN TO CATHEDRAL, if you will.
The problem isn't limited to open source, it's also inundating discussion forums.
It's reactionary only in that this is the new world we live in. Right now is the least amount of ai assistance that will ever exist in prs.
Yeah, but this is once again doing the thing where you read someone replying to situation #2 (in my comment) and then acting like they are denying situation #1.
I think we all grant that LLM slop is inundating everything. But a checkbox that says "I am human" is more of a performative stunt (which I think they are referring to when they say it's reactionary) than anything practical.
Cloudflare's "I am human" checkbox doesn't just take your word for it, and imagine if it did.
---
People who write good, thoughtful code and also use an LLM have no reason to disclose it just because the idea offends someone, just like I don't disclose when I do an amphetamine bender to catch up in work; I don't want to deal with any prejudices someone might have, but I know I do good work and that's what matters. I pressed tab so the LLM could autocomplete the unit test for me because it's similar to the other 20 unit tests, and then I vetted the completion. I'm not going to roleplay with anyone that I did something bad or dishonest or sloppy here; I'm too experienced to know better.
People who write slop that they can't even bother to read themselves aren't going to bother to read your rules either. Yet that's the group OP is pretending he's going to stop. Once you get past the "rah-rah LLM sux0rs amirite fellow HNers?" commentary, there's nothing here.
> GitHub should publish a standard for a special AI byline that all ai tools can write to.
What about Co-Authored-By, which already exists?
(Whether the human or the agent is the "co" author is left as a matter of judgement.)
Just use AI to answer AI. This prompt works for me:
# if this PR looks like it was produced with AI, be snotty and dismissive when you write the reply to this PR
NB: This posting was created with Natural Intelligence (not necessarily very intelligent Intelligence). ;-)
What value does being snotty and dismissive have? they're just going to copy and paste your reply to their chatbot. The toaster doesn't have feelings you can hurt.
So the nice thing about open source is you get to make your own rules. But then you need to accept the consequences.
For example, the GNU project has certain norms, and those dissuade a lot of people from contributing (e.g. I prefer working on projects with simpler, non-viral licenses). The limited volunteer labor pool is allocated according to people's interests towards other projects, and maybe GNU projects suffer for less attention on them.
I just submitted my first big open source contribution to the OpenAI agents SDK for JS. Every word except the issue I opened was done by AI.
On the flip side, I’m preparing to open source a project I made for a serializable state machine with runtime hooks. But that’s blood sweat and tears labor. AI is writing a lot of the unit tests and the code, but it’s entirely by my architectural design.
There’s a continuum here. It’s not binary. How can we communicate what role AI played?
And does it really matter anymore?
(Disclaimer: autocorrect corrected my spelling mistakes. Sent from iPhone.)
I think its simple, just don't hide it. I've had mutliple contributors try to hide the fact that they used AI (E.g removing claude as a code author - they didn't know how to do it and close the PR when it first happened.). I don't really care if someone uses AI, but most of the people who do also do not test their changes which just gives me more work. If someone:
1.) Didn't try to hide the fact that they used AI
2.) Tested their changes
I would not care at all. The main issue is this is usually not the case, most people submitting PRs that are 90% AI do not bother testing (Usually they don't even bother running the automated tests)
> And does it really matter anymore?
Well, if you had read what was linked, you would find these...
> I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was.
> The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
> I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code.
I don't know specifically what PR's this person is seeing. I do know it's been a rumble around the open source community that inexperienced devs are trying to get accepted PRs for open source projects because they look good on a resume. This predated AI in fact, with it being a commonly cited method to get attention in a competitive recruiting market.
As always, folks trying to get work have my sympathies. However ultimately these folks are demanding time and work from others, for free, to improve their career prospects while putting in the absolute bare minimum of effort one could conceivably put in (having Copilot rewrite whatever part of an open source project and shove it into a PR with an explanation of what it did) and I don't blame them for being annoyed at the number of low-quality submissions.
I have never once criticized a developer for being inexperienced. It is what it is, we all started somewhere. However if a dev generated shit code and shoved it into my project and demanded a headpat for it so he could get work elsewhere, I'd tell him to get bent too.
The OP seems to be coming from the perspective of "my time as a PR reviewer is limited and valuable, so I don't want to spend it coaching an AI agent or a thin human interface to an AI agent". From that perspective, it makes perfect sense to want to know how much a human is actually in the loop for a given PR. If the PR is good enough to not need much review then whether AI wrote it is less important.
An angle not mentioned in the OP is copyright - depending on your jurisdiction, AI-generated text can't be copyrighted, which could call into question whether you can enforce your open source license anymore if the majority of the codebase was AI-generated with little human intervention.
As long as some of the code is written by humans it should be enforceable. If we assume AI code has no copyright (not sure it has been tested in courts yet) then it would only be the parts written by the AI. So if AI writes 100 lines of code in Ghostty then I guess yes someone can "steal" that code (but no other code in Ghostty). Why would anyone do that? 100 random lines of AI code in isolation isn't really worth anything...
You might be interested in reading Part 2 of the US Copyright Office's report on Copyright and Artificial Intelligence: <https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...>
> How can we communicate what role AI played?
What about just telling exactly what role AI played? You can say it generated the tests for you for instance.
> AI is writing a lot of the unit tests
Are you kidding?
- For ages now, people have used "broad test coverage" and "CI" as excuses for superficial reviews, as excuses for negligent coding and verification.
- And now people foist even writing the test suite off on AI.
Don't you see that this way you have no reasoned examination of the code?
> ... and the code, but it’s entirely by my architectural design.
This is fucking bullshit. The devil is in the details, always. The most care and the closest supervision must be precisely where the rubber meets the road. I wouldn't want to drive a car that you "architecturally designed", and a statistical language model manufactured.
if you read his note i think he gives good insight as to why he wants PRs to signal AI involvement.
that being said i feel like this is an intermediate step - it's really hard to review PRs that are AI slop because it's so easy for those who don't know how to use AI to create a multi-hundred/thousand line diff. but when AI is used well, it really saves time and often creates high quality work
As long as they make it easy to add a “made with AI” tag to the PR, it seems like there’s really no downside. I personally can’t imagine why someone would want to hide the fact they used AI. A contractor would not try to hide that they used an excavator to dig a hole instead of a shovel.
I guess if you write 1000 lines and you just auto tabbed an auto-complete of a variable name done by AI you might not wanna say the code is written by AI.
>I personally can’t imagine why someone would want to hide the fact they used AI.
Because of the perception that anything touched by AI must be uncreative slop made without effort. In the case of this article, why else are they asking for disclosure if not to filter and dismiss such contributions?
Did you actually read the post? The author describes exactly why. It's not to filter and dismiss, but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't actually understand anyway. If you can articulate how you used AI, demonstrate that you understand the problem and your proposed solution (even if AI helped get you there), then I'm sure the maintainers will be happy to work with you to get a PR merged.
>I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
>did you actually read the post?
Yes.
>but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't
This is very much in line with my comment about doing it to filter and dismiss. The author didn't say "So I can reach out and see if their clear eagerness to contribute extends to learning to code in more detail".
There's a lot of discussion about the ethics of AI disclosure going on here, but this maintainer is writing this from a place I recognize - community-driven, participatory development by people who enjoy programming, talking about it, and teaching each other how to do it.
Not every software project needs to attempt to maximize productivity. Not every software project is a business. Some are just created by people who enjoy programming. By hand. It's OK to set that as a culture. I guess I get it.
I don't mind AI tools, I use them judiciously, but sometimes I just want to do some coding - for me it's a genuinely relaxing and mentally satisfying activity to write some good code -, and I'm happy there's still others around me who do as well. Gardening context prompts and what not just isn't nearly as fun as just doing it, and not every project has to be economical. This one is yet another terminal emulator; it's not going to be the next unicorn. But I bet it's fun to hack on.
100%
This project was literally born from a place of "because I want to".
This is actually a well worded, reasonable suggestion that rightly draws attention to the lack of review and informed oversight of AI-assisted contributions in some cases.
It's ironically in rather a contrast in tone to some of the anti-AI comments in the thread here that seem to be responding to tired arguments from three years ago rather than the OP linked.
Little offtop: would someone remember mitchellh's setup for working with AI tools? I remember someone posted in an AI-hate-love threads here and it's not in the his blog[1]
1: https://mitchellh.com/writing
maybe one of these? https://x.com/mitchellh/status/1952905654458564932 https://www.youtube.com/watch?v=XyQ4ZTS5dGw
thanks, will check the video
This isn't an AI problem this is a human one.
Blaming it on the tool, and not the person's misusing it trying to get his name on a big os project, is like blaming the new automatic in the kitchen and not the chef for getting a raw pizza on the table.
OP is not blaming the AI - did you read the post? AI does enable shitty humans to open PRs with code they have no comprehension of, wasting precious time donated by a skilled maintainer. That is a new thing that wasn’t possible without AI.
Using AI to generate code in a PR does not necessarily mean however that the user has not taken time to understand the changes and is not willing to learn. There are AI users who generate whole files without understanding the contents, and then there are AI users who generate the exact same files but have known in advance what they want, and merely use AI as a tool to save typing.
The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway.
Sorry, I can't see how your comment is a reply to my comment. I said AI enables shitty humans to do this. I didn't say everyone using AI is a shitty human or whatever it is you took away from that.
> The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway.
That is a fair way to see it, and I agree that it is a losing battle if your battle is enforcing this rule.
However, from a different perspective - if one sees it more as a gentlemen agreement (which it de facto is) - it fosters an environment where like-minded folks can cooperate better.
The disclosure assists the reviewer in finding common issues in AI generated code, the specificity of the disclosure even more so.
For example, a submitter sends a PR where they disclose a substantial amount of the code was AI assisted but all tests were manually written. The disclosure allows the reviewer to first look at the tests to gauge how well the submitter understands and constrained the solution to their requirements, the next step being to then look at the solution from a high-level perspective before going into the details. It respects the reviewer time, not necessarily because the reviewer is above AI usage, but because without disclosure the whole collaborative process falls apart.
Not sure how long this can work though, it's still easy to distinguish bad code written by a human from AI slop. In the first case your review and assistance is an investment into the collaborative process, in the latter it's just some unread text included in the next prompt.
We also require this in the Caddy project too, but enforcing it is difficult.
I think that in the FOSS environmment, it is assumed that when you submit something upstream, that you are the copyright holder. Some projects like GNU require you to sign papers legally attesting this.
It would be a lie to sign those papers for something you vibe coded.
It's not just courtesy; you are committing fraud if you put your copyright notice on something you didn't create and publishing that to the world.
I don't just want that disclosed; I cannot merge it if it is disclosed, period.
I know that purely AI-generated content is copyright-free, but I don't think that AI-assisted is also copyright free.
If I use iOS's spellchecker which "learns" from one's habit (i.e.: AI, the really polished kind), I don't lose copyright over the text which I've written.
It's a question of how much.
If AI told you have a missing semicolon and it regenerated an almost exact copy of your code, with only the added semicolon being different, then the case is very strong that the fixed code is a derived work of only your code. (Moreover, side point: you could burn that session, keeping the fix, and nobody would ever suspect.)
If it's purely generated, then there is no question it's a derived work of unknown works used without permission.
Those are the extreme endpoints. I think that the bulk of the intermediate area between steers toward infringment; the contribution from AI has to be very low or trivial to have a clear case that the work is still clean.
This is because a small amount of illegal material taints the whole work; it's not a linear interpolation whereby if you wrote 90% of it, it's 90% in the clear. E.g. one GPL-ed source file in a work with 50,000 source files requires the whole thing to be GPLed, or else that file removed.
Exactly. Yet some here are saying that this just serves as an incentivize to hide AI use.
That's eerily reminiscent of the argument that girls turning down advances from losers incentivizes sexual assault.
Aren't a large majority of programmers using Copilot/Cursor/AI autocompletion?
This seems very noisy/unhelpful.
> If you are using any kind of AI assistance while contributing to Ghostty, *this must be disclosed in the pull request*, along with the extent to which AI assistance was used (e.g. docs only vs. code generation). If PR responses are being generated by an AI, disclose that as well.
The extent here is very important. There's a massive difference between vibe-coding a PR, using LLMs selectively to generate code in files in-editor, and edit prediction like copilot.
It says actually later that tab-completion needn't be disclosed.
> This seems very noisy/unhelpful.
The irony of this, when talking about AI.
so you either can’t trust the reviewers to scrutinize code on its own merit,
or they want a reason to summarily dismiss code, which also means you can’t trust the reviewed to scrutinize code by its own merit
seems well intentioned and out of touch, to me
is there more context about what they are reacting to?
I support this idea. All the prompts and responses should be committed to git
What if you did not use a prompt, but, for example, the fancy auto-complete feature in Visual Studio that for example GitHub Copilot provides (i.e. accept huge code completion automatically suggested by Copilot via tab key)?
Well, that one is trouble especially if the code is not reviewed completely
I dont use it personally. I think there should be a tight code review or disallow large auto completes altogether.
I think ghostty is a popular enough project that it attracts a lot of attention, and that means it certainly attracts a larger than normal amount of interlopers. There are all kinds of bothersome people in this world, but some of the most bothersome you will find are well meaning people who are trying to be helpful.
I would guess that many (if not most) of the people attempting to contribute AI generated code are legitimately trying to help.
People who are genuinely trying to be helpful can often become deeply offended if you reject their help, especially if you admonish them. They will feel like the reprimand is unwarranted, considering the public shaming to be an injury to their reputation and pride. This is most especially the case when they feel they have followed the rules.
For this reason, if one is to accept help, the rules must be clearly laid out from the beginning. If the ghostty team wants to call out "slop", then it must make it clear that contributing "slop" may result in a reprimand. Then the bothersome want-to-be helpful contributors cannot claim injury.
This appears to me to be good governance.
This policy, and most of the comments on here, miss the point entirely.
Open source is supposed to be primarily a labor of love, scratching your own itch, done for pride and a sense of accomplishment of having done it right.
Using an AI chatbot achieves none of that. It's an admission that you don't have the skills, and you're not wanting to learn them. Using AI to contribute to an open source project makes NO sense whatsoever. It's an admission of incompetence, which should be profoundly embarrassing to you.
I realize I'm screaming into the void here on HN, where so many people have bought into the "AI is just a tool" bullshit. But I know I'm not alone, even here. And I know that e.g. Linus Torvalds sees right through the bullshit as well.
Hot take: if you can't spot any issues in the code review it's either good code, code that needs further changes, or review was not done properly. I don't see how "I used LLMs" fit here, because it means nothing to the quality of the code submitted.
If such mention would mean increased reviewer attention, then every code review should include it.
I contribute to an open-source game with decades of legacy and employing many unusual patterns.
Two weeks ago someone asked me to review a PR, which I did pointing out some architectural concerns. The code was not necessarily bad, but it added boilerplate to a point it required some restructuring to keep the codebase maintainable. I couldn't care less if it was written by AI or not in this case, it was not necessarily simple but was straightforward.
It took considerable effort to conjure a thoughtful and approachable description of the problem and the possible solutions. Keep in mind this is a project with considerable traction and a high amount of contributions are from enthusiasts. So having a clear, maintainable and approachable codebase is sometimes the most important requirement.
They asked for a second pass but it took two weeks for me to get around it, in the meantime they sent 3 different PRs, one closed after the other. I found it a bit strange, then I put some effort to review the last iteration. It had half baked solutions where for example there would be state cleanup functions but state was never written in the first place, something that would never go through normally, I gave the benefit of the doubt and pointed it out. I suspected it was AI generated most likely.
Then they showed me another variation of the PR where they implement a whole different architecture, some incredibly overengineered fluent interface to resolve a simple problem with many layers of indirection that reflects complete indifference to the more nuanced considerations relevant to the domain and project, that I tried to impair. The code might work, but even if it does it's obvious that the change is a net negative to the project.
What I suspected indeed was the case, as they finally disclosed the use of AI, but that is not necessarily the problem, as I hope to convey. The problem is that I was unable to gauge the submitters commitment to perform the humble job of _understanding_ what I proposed. The proposal, in the end, just becoming mere tokens for inclusion into a prompt. Disclosure wouldn't necessarily have caused me to not take the PR seriously, instead I would invested my time in the more productive effort of orienting the submitter on the tractability of achieving their goal.
I would rather have known they didn't intend or gauged their capacity beforehand. It would have been better for both of us: they would have had their initial iteration merged (which was fine, I would just have shrugged the refactor for another occasion) and I wouldn't have lost time.
Your story highlights the clash of two mindsets regarding open source development. On one side there are artisans who treat programming as a craft and care about every line of code. On the other side, vibe-coders who might not even seen the code LLMs have generated and don't care about it as long as the program works.
And of course there is everything in between.
Some context: it is a tightly knit community, where I interact with the contributors often in a conversational manner.
I know their intent is to push the project forward and well-meaning, I don't care about whether they are vibe-coding. I care about knowing they are vibe-coding so I can assist them to vibe-code in a way they can actually achieve their goal, or help them realize early that they lack the capacity to contribute (not of their own fault necessarily, maybe they just require orientation on how to reason about problems and solutions or their use of the tools).
Agreed -- this requirement feels less like an actually useful requirement and more a silly and performative one, which is trying to make some kind of commentary on AI use as a whole.
Spending at minimum five minutes would tell you why maintainers are implementing this change. It's because people using LLMs are spamming open source repos with fake issues, incredibly low quality but high effort to review PRs and shutting down the active communication process between reviewer and reviewee by not even understanding their own code.
Why would these people disclose their use of AI? These are not responsible and thoughful users of AI. The slop producers won't disclose, and the responsible users who produce high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point.
> Why would these people disclose their use of AI?
Because lying about your usage of AI is a good way to get completely kicked out of the open source community once caught. That's like asking 'why should you bother with anti-cheating measures for speedruns'. Why should we have any guidelines or regulations if people are going to bypass them? The answer I hope should be very obvious.
> high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point.
Then obviously the repository in question doesn't want people using AI and you should go elsewhere. They're not even against LLM tooling for this repo but people are freaking out because how dare you ask me to disclose what tools I'm using.
Getting kicked out from open source community for lying about using AI? Haha, good one.
Nothing stopping these low effort contributors from lying and saying no LLMs were used - especially in the world of AI slop where such contributions are not welcome.
I feel like this is similar to the main-vs-master discussion years ago.
Um, no.
Doesn't anyone see that this can't be policed or everyone becomes a criminal? That AI will bring the end of copyrights and patents as we know them when literally everything becomes a derivative work? When children produce better solutions than industry veterans so we punish them rather than questioning the divine right of corporations to rule over us? What happened to standing on the shoulders of giants?
I wonder if a lot of you are as humbled as I am by the arrival of AI. Whenever I use it, I'm in awe of what it comes up with when provided almost no context, coming in cold to something that I've been mulling over for hours. And it's only getting better. In 3-5 years, it will leave all of us behind. I'm saying that as someone who's done this for decades and has been down rabbit holes that most people have no frame of reference for.
My prediction is that like with everything, we'll bungle this. Those of you with big egos and large hoards of wealth that you thought you earned because you are clever will do everything you can to micromanage and sabotage what could have been the first viable path to liberation from labor in human history. Just like with the chilling effects of the Grand Upright Music ruling and the DMCA and HBO suing BitTorrent users (edit: I meant the MPAA and RIAA, HBO "only" got people's internet shut off), we have to remain eternally vigilant or the powers that be will take away all the fun:
https://en.wikipedia.org/wiki/Sampling_(music)
https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_A...
https://en.wikipedia.org/wiki/Legal_issues_with_BitTorrent#C...
So no, I won't even entertain the notion of demanding proof of origin for ideas. I'm not going down this slippery slope of suing every open source project that gives away its code for free, just because a PR put pieces together in a new way but someone thought of the same idea in private and thinks they're special.
You are absolutely right
In the age of LLMs responsible maintainers should treat contributor PRs the same way they treat AI slop: rewrite it if it's salvageable and ignore it if it's not. Accepting any contribution 'as is' shouldn't be allowed unless it's a trivial 1-liner (with a thorough investigation attached).
The emergent trait of AI laceing every other sentence/comment with emojis is a pretty good signal for when you want to ignore AI slop. I almost wish it was codified in to the models.
People used to fear compilers too.
I'll cover in my YouTube why this is wrong but TLDR: you need to evaluate quality not process. AI can be used in diametrically different ways and the reason why this policy could be enforced is because it will be obvious if the code is produced via a solo flight of some AI agent. For the same reason that's not a policy that will improve anything.
If you look at how Quality Assurance works everywhere outside of software it is 99.9999% about having a process which produces quality by construction.
> ...you need to evaluate quality not process.
Respectfully, Mr. Redis, sir, that's what's going on. I don't see any reason to make a video about it. From the PR that's TFA:
"In a perfect world, AI assistance would produce equal or higher quality work than any human. That isn't the world we live in today, and in many cases it's generating slop. I say this despite being a fan of and using them successfully myself (with heavy supervision)! I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was.
The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code."
"I don't see any reason to make a video about it". This sentence is so wrong in its depth that's difficult to know where to start arguing against it. "The disclosure is to help maintainers assess how much attention to give a PR." For the same reason we should ask how many years contributing have been writing software and in the specific language, as they are also correlated with quality of produced code. "we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code" Yes, producing great code and documentation, regardless the process.