May be we can require authors to declare use of AI-generated content in the commit message using some fixed tags like “Contains AI generated code: Yes/No” and if the patch is particularly long as well then the reviewers can request the authors to split it into smaller units (like the way they today).
Large patches are anyways discouraged unless they are coming from well-known reliable authors. So, I don’t see a situation where some one can simply put large pieces of immature code in a PR and reviewers are forced to spend significant amount of time in reviewing it.
I have two primary concerns. We know some AIs are trained on copyrighted or private materials, but it’s difficult to know which AIs and how likely the chance is that one spits out copyrighted code. Us accepting code without the rights to it is a significant risk to the community and the LLVM Foundation. I assume everyone is contributing in good faith, but it is easier for someone to accidentally contribute code they don’t have the rights to when using AI-generated code. So those reviews need more oversight for me to feel comfortable accepting them, and that’s a burden on me as a reviewer.
Secondarily, I do not trust an AI to generate good code when generating a large quantity of it. I’m not worried about small contributions where an AI generates a few lines, but I am worried when an AI generates most of a large patch. For a larger contribution, when a human wrote the code, I expect they understand how the code works better than when they let an AI write the code for them. That means the amount of time spent going back-and-forth when problems are found is going to be significantly higher when dealing with AI-generated code, which is also a burden on me as a reviewer.
If a reviewer doesn’t notice an AI was used, it’s not really a problem that an AI was used. If you notice there was AI-generated content, it puts more burden on the reviewer to verify the validity of the patch. Saying “we don’t accept AI-generated code” makes it far easier for reviewers to handle the situations where use of AI is a problem in practice while still “allowing” AI to be used in the situations where we don’t even know it was being used. It’s not ideal in terms of a policy, but it seems practical to me.
1 Like
Thank you for keeping this in mind!
I think we have some common ground in terms of what we want to allow: if use of AI goes basically unnoticed during a review, it likely wasn’t a problem because it’s probably not copyrighted material (any more likely to be copyrighted than any other kind of contribution) and it didn’t introduce a reviewer burden. But I think it’s easier from a policy perspective to go the other way and have a restrictive policy on tool use but encourage reviewers and contributors to exercise good judgement around AI use in practice (which we have no practical way to prevent other than via social contract).
If we’re permissive, the burden is on the reviewers to notice copyrighted material or have more awkward conversations when AI use is causing problems. The LLVM Foundation is also accepting slightly more risk of copyrighted material being committed (I expect more people to introduce copyrighted code accidentally via AI than to do so intentionally).
If we’re restrictive, the burden is on contributors to make sure that AI-generated code isn’t going to be noticed during review. This gives reviewers an easy way to handle problems (“Our policy is to not accept AI-generated code, I’m closing this review.”) Also, the LLVM Foundation is accepting slightly less risk of copyrighted material being committed and can at least point to a policy that says AI wasn’t supposed to be used (in addition to our existing policy around copyright). Note, I’m not expecting reviewers to try to spend effort guessing at whether an AI was used; I’m expecting reviewers to continue to assume the code is all written by a human and only need to fall back on the policy when there’s an actual problem noticed during review.
It’s a bit uncomfortable because it’s somewhat of a “wink wink” situation where we’re allowing AI to be used if it’s sneaky enough, but I think it still strikes a good balance while hopefully protecting the foundation and reviewer resources better.
1 Like
I might agree with you until this sentence: it’s not clear to me why there is more burden to you. You point to an added risk from the use of AI for contributors, but why would it somehow become your responsibility as a reviewer to look into whether the author of the patch isn’t infringing on someone’s copyright? As far as I know it isn’t the case right now and AI does not change this.
How so? I don’t follow why the fact that the AI does a good job at imitating a human would be less risky in terms of copyright infringement?
Typically reviewers can reasonably ‘trust’ that if the code was authored by a contributor that it was personally reviewed by that author, and that they wrote it for the project: that is, they aren’t acting in ‘bad faith’ regarding copyright.
I don’t have an ability to judge whether the AI used is even AWARE of copyright, let alone whether it would avoid it as a ‘good faith’ effort here. Therefore, if I KNOW an AI is involved, I have to assume that the author is acting in the equivalent of “bad faith” (acknowledging of course, that computers cant act in good/bad faith), thus I VERY MUCH need to be more careful when reading it. Bugs, hallucinations, truth of comments, and yes, copyright.
2 Likes
I am of the opinion that accepting any contributions involving generative ML is rather risky for the time being. From a cursory search, there is no settled case law in the United States on how copyrights are handled relating to the output of such programs, and as such, no guarantee that any output is legally okay.
If you ask me, it would be wise to ban any contributions that these models had any part in, at least for the time being. There is just too much dust that has yet to settle.
I’m unsure of the hallmarks within code itself, but I’ve found ML-generated text to be easily detectable by writing style—in my opinion, any text in or relating to a commit with that sort of writing style should raise heavy scrutiny.
As for the use of such models in forums or issues: please no. It always either hallucinates or obfuscates meaning, and adds useless padding to text. It’s one thing if you translate languages, but if you’re trying to use it to “make your text longer” or “make it clearer”, it will just make it worse.
5 Likes
Basically, I assume everyone is contributing in good faith, and that means I assume folks are not copy/pasting code from elsewhere and submitting it as a PR. So I don’t try to actively identify copyright violations today unless the contributor gives me a reason to think there may be an issue (like a commit message or comment saying “I copied this from <wherever>”), and I think that’s the way other reviewers generally operate as well. But even when assuming good faith with AI-based contributions, nobody involved has a reasonable way to know if there is accidentally copyrighted material being contributed but we do know that AIs have been trained on copyrighted materials.
If reviewers don’t worry about copyright issues today (unless given a reason to worry), then not knowing an AI was used is no different from the status quo. But for known AI-based contributions, that gives us a reason to worry just like someone saying “I copied this from somewhere else”, which means it requires more oversight during review.
2 Likes
Is it part of most people’s development? My feeling is that it isn’t, but I might be living in a bubble.
I think knowing to what extent active contributors already use AI would be good, to help guide a pragmatic policy. E.g. if 90% of contributors already use AI, then it seems to be working and we’d increase the barrier a lot if we had a hard line on AI, but if only 10% of contributors use AI, we could hold a pretty hard line without affecting contributions much.
I haven’t tried the AI tools: how well do they work in LLVM? Do they use LLVM idioms, or are they limited to IntelliSense replacement?
2 Likes
Same question here.
If you’re living in a bubble, it has to accommodate me and 4 more regular Clang contributors.
1 Like
I’m in favor of allowing AI assisted contributions with the caveat that normal review and contributions standards apply. I don’t think we need to set a different standard here. A gentle reminder to the effect that the patch author is responsible for the quality and licensing of all code in a review (regardless of source) might be warranted, but that’s more a reminder than a new or different policy.
One challenge I see with trying to do anything restrictive here is that it forces us to define what we mean by AI. As a real example recently, I did a bit of work around constant materialization using a program synthesis bases tool I wrote based on code a grad student had released. Was that AI? I don’t want to get into the game of trying to answer that question precisely.
(For the record, that tool generated assembly patterns which I then manually wrote code for - but it certainly could have generated the code if I’d bothered to spend a bit more effort.)
@preames and @mehdi_amini’s comments above broadly match my feelings on the matter. I don’t think we should ban using AI tools in the process of development. For starters, I expect in many instances, we wouldn’t be able to tell whether one is used or not anyway: there are plenty of low-quality monolithic PRs that I’ve seen and rejected during review, pending cleaning and splitting into more manageable, higher quality PRs, and I have no reason to believe that all or even any of these were generated by AI. So how are we to tell whether something has been generated using AI? What even is AI? If a developer has used an AI tool to start them off with a piece of work, how do they know when they’ve done enough to make it “not generated by an AI tool”. I don’t routinely use AI myself, but I have on a very limited number of occasions used it in other work to help me figure out how to do something somewhat complex using the Windows API. I sometimes may have copied snippets of that code, or written my own code in a way that was inspired by the generated example. Does that make it “AI generated”?
On the note of copyright, I am not a lawyer (and I doubt anybody else is in this thread). I think we should leave it to the lawyers to determine whether or not something is in breach of copyright. If we’re concerned about whether a specific contribution is, I suppose it could be escalated to the LLVM legal counsel. However, I don’t think this is in any way limited to “AI generated” stuff. If there’s a real concern that AI generation tools breach copyright, perhaps it should be up the LLVM Foundation to make that determination (presumably in conjunction with their legal counsel).
FWIW, +1 to the original policy suggestion, with the suggested FAQ.
None of the developers I know use even copilot. Apparently, it’s mostly newer developers using AI but I’d certainly consider myself to be new having only started full-time within the past year. I’ve had only negative experiences with using AIs for code but I haven’t tried in a while.
I think it’s hard enough to prove that code is written by AI for there to be much of a difference between rejecting a PR due to low quality (or inability to explain the code, etc) and rejecting due to AI use. I do think having a general policy of keeping AI use to autocompletion would be reasonable.
I don’t see a problem with labeling LLVM as containing some code written by AI if there is a good (legal?) reason why it needs to be done. I’d be more upset if the requirement was just to label it as AI generated with no qualifiers (which is something I’ve seen on videos and pictures on some platforms even if they contain only a small amount of AI generated content).
(IANAL, this is not legal advice, opinions are expressly my own, etc.)
It seems like the answer is unequivocally “yes, you should consider LLVM to contain AI-generated content”. Even if we change some policy now I don’t imagine we will be going back to every contributor and getting them to attest that they didn’t contribute any AI-generated content? Even if we did we could only upgrade our response to “maybe; every contributor claims to have never contributed AI-generated content, but people may have lied ¯\_(ツ)_/¯”.
If the concern is not about AI-generated stuff per se, but actually about negligently accepting contributions of infringing material in a way that opens up liability for the maintainers then we could borrow the kernel’s Developer Certificate of Origin approach, and just try to push the question (and as much of the liability associated with it) onto the contributor.
(See also Submitting patches: the essential guide to getting your code into the kernel — The Linux Kernel documentation and Developer Certificate of Origin.)
As a reviewer on behalf of the project I personally don’t consider the origin of a patch whatever, inclusive of whether accepting it could constitute some form of copyright infringement. That seems like an impossible ask.
A much more feasible ask is for reviewers to enforce a CLA/CTA/DCO.
1 Like
Hey folks, I got busy and left this alone for several months, but I added a paragraph to my policy here that I believe incorporates the feedback I got here to encourage contributors to take ownership of any generated code contributions:
While the LLVM project has a liberal policy on AI tool use, contributors are
considered responsible for their contributions. We encourage contributors to
review all generated code before sending it for review to verify its
correctness and to understand it so that they can answer questions during code
review. Reviewing and maintaining generated code that the original contributor
does not understand is not a good use of limited project resources.
See the full text in the PR.
I think I’ve gotten approval on the policy in this RFC, but I’ll wait until Monday to land this in case folks have more feedback. Folks can feel free to propose edits to the non-policy encouragement text outside of the RFC process if they like.
To respond to some points made here since I’ve posted:
Thanks for the link, I actually know very little about other OSS project processes. I think the Linux process here is very close in spirit to what we want to do. I see the Linux process involves adding a formal Signed-off: tag to every commit, which feels like extra paperwork, unless it’s mechanized as part of the review process. It’s something to think about going forward.
4 Likes
I agree the “Signed-off” bit seems like administrivia. I assume it is there to make consent explicit, but it may not be required if LLVM adopts a similar policy (again IANAL).
I think EULAs can be implicitly accepted through use of a product, so maybe analogously we could have a DCO whose “sign off” is via the act of submitting a patch for review?
I think that’s the intention of our existing policies, but we should write it down more explicitly. Maybe we could create some GitHub PR automation that requires first-time contributors to check a button to record explicit agreement that subsequent patch contributions include an implicit DCO.
Separately, I’m going to merge the AI policy doc. Thanks for the input!
Worth noting that NetBSD assumes AI generated code is tainted and requires explicit disclosure and more rigorous review: NetBSD Commit Guidelines
I think this is worth keeping in mind if this discussion comes up again.
1 Like