Repository navigation
[MNT]: Should we request contributors to declare usage of AI? #30848
Description
Summary
Proposal: Add checkboxes
- issue/PR messages has been created with the help of AI
- Code has been created with the help of AI. - Maybe even split in finding the solution/implementing the solution.
Motivation: Prior to AI times, the messages and code quality gave a strong hint on the experience of contributors, which influences the way I review, both in communication and in assumption how well the topic is thought trough. This indicator is gone now that AI will produce good looking results. While stating AI use cannot bring it back, for me it would trigger a warning lamp to more carefully review the logic. (And possibly sometimes put less effort in a review and instead request additional clarification early - if something looks odd, I would spend less time in trying to figure out whether this is actually right if AI was used)
Reacted by Ruth Comer, hannah and Théo SZANTO
Activity
rcomer commentedon Dec 13, 2025
I would also favour adding something like "please state what was done by AI and what was done by a human" for code changes.
timhoffm commentedon Dec 13, 2025
I'm a bit undecided how formally structured/free we want this statement.
While I found this statement from a recent PR very helpful
I'm somewhat concerned that less experienced contributors will not provide useful information in free text.
story645 commentedon Dec 14, 2025
I suspect that the folks we're trying to get this information from are feeding our templates to AI, so is there a way we can engineer this prompt so that AI will respond accordingly?
timhoffm commentedon Dec 14, 2025
We might get AI to respond like this:
And then a human reviewer will change the second one.
We still have to think about the exact wording / questions.
Semi-OT: Since people feed our stuff to AI, we may be able improve results by updating our instructions.
rcomer commentedon Dec 14, 2025
I guess if we just have the tick box, we can always ask follow-up questions if we suspect there was no human intervention in the implementation.
ianhi commentedon Dec 14, 2025
I think yes. Given the publicity and the number of PRs matplotlib gets not doing so will result in an unfortunate increase in maintenance burden/reviewer burnout. But the phrasing might require some thought. See below
The following two responses below raise some concerns. But I want to be clear I am not opposed to the goal here, just raising potential complications in how to deal with this tough problem.
I'm starting to assume by default that every PR I review has some AI influence on it. Quite literally everyone I know who is still in some form of school regularly uses AI while coding, and I use it daily. It's tough to decide where to draw the line when defining that split. Having claude write code is clearly AI, but where does AI powered auto-complete fall? For text descriptions (e.g. docstrings, docs, or even a PR descriptions) I have sometimes written out a fairly detailed skeleton, riddled with spellign and grammar errors, passed it through my claude session once to clean it up and used that . (I have not done for this comment )
Similarly sometimes I use claude as a high powered grep to trace through an execution graph and have a back and forth to determine the architecture of a fix. I think I would probably still consider the core of the idea to be mine with thinking accelerated by claude. But it is also no longer just my thoughts.
In #30844 how would I answer this? I pointed at line numbers and told claude how i wanted things changed. Is that being done by AI, or is that me (a certified human) using claude as powerful IDE?
What I'm trying to get at above is that I'm not sure that there is a simple question like "written by AI" that can cover all cases. But that having a question like that that may be a small friction for an experienced (but new) contributor who uses AI in the ways I described above. Though I suspect that that is an ok tradeoff in order to catch the ones where someone points an AI at an open issue and says "please solve".
I'm not sure AI would always read the template? e.g. if the users asks them to make a PR I think it would potentially use the gh cli to open a PR, which I think bypasses seeing the template.
But if a user feeds the template into an AI then adding a line here is a very nice idea.
Have you thought about adding an AGENTS.md, That could also be a chance to leave instructions about this. For example the Xarray "Github Interaction Guidelines" section of CLAUDE.md:
https://github.com/pydata/xarray/blob/main/CLAUDE.md#github-interaction-guidelines
This is also a place where dev instruction for pixi (especially called out in agents.md) could be very helpful. I find that claude really struggles to use conda envs due some shell trickery that I don't really understand. In contrast it is able to use pixi very well. This let's you get into a very effective feedback where you can direct the high level changes and it is able to run code to check things, or to run the test case you've given it to see if the implementation is working. Which might result in AI powered PRs having more tightly integrated and well written code.
Can the new contrib bot add a question here? Ideally in a non-judgemental way to keep the vibe of the bot the same.
story645 commentedon Dec 15, 2025
Something like, "Please let us know if (and how) you use AI, it'll help us give you better feedback on your PR." ?
Which is the truth of the matter really.
timhoffm commentedon Dec 15, 2025
@ianhi Thanks for your detailed answer 💯. You have some very good points. There are various ways to use AI with substantially different impact, and communicating how AI was involed clearly is difficult. To narrow this down, I'd say that it's relevant whether significant portions of (1) Solution idea / architecture and (2) Implementation are AI based.
But actually, the more fundamental question is whether the author has thought the solution through and cross-checked the implementation (including the ability to do that) - no matter whether AI was involved or not. Unfortunately that's a bit harder to ask. Our policy on AI useage goes somewhat in that direction. What we really would like to know is
But I suspect both boxes will be always be ticked whether true or not, so not much help :(. Maybe we can find better questions? I think careful and precise wording is essential here.
Re contrib bot: One thing is that the bot will only pop in on the first PR. But maybe that can be changed. The more fundamental issues are that it's delayed extra work. It's a bit like these annoying customer questionaires: you have fiddled through the order process / PR and think you have done it and a short time later you get the request that it would be super helpful if you answer additional questions. The motivation to do that or do that seriously is small.
jklymak commentedon Dec 16, 2025
I think improvements are improvements, and it doesn't really matter how they are created so long as it respects everyone's time.
I think there have been some recent examples of disrespectful use, where contributors are using AI and using review comments to the drive the AI, with little expert input from themselves. Thats wildly inefficient as the Reviewer could more quickly drive the AI themselves.
I'm not sure how you discourage that, other than simply not interacting with such contributions. The problem is that AI-generated comments are verbose enough you can't easily tell for sure if you are dealing with a bot or someone who has just used AI to clean up their English. I think the use of AI text should reflect that it's unacceptable to simply use review comments as prompts to AI and that Contributors are expected to understand what they are doing.
timhoffm commentedon Dec 17, 2025
Due to recent events (#30872, #30873, #30876), I propose to set a policiy that a new contributor cannot make more than 1 PR at a time (or per day). At minimum in our contributors guide, but if technically enforceable possibly even that.
Another proposal for discussion:
Pull request messages must not be created with the help of AI
Rationale: The PR is the main communication to reviewers. It should carefully lay out what the PR does and why. With a PR, an author asks for reviewer time. It can be expected that they communicate as best as possible and not use pre-fabricated (and possibly unvalidated) texts from an AI. This also ensures that the author has to understand the issue and his solution.
Countering possible counter-arguments:
story645 commentedon Dec 18, 2025
Is it worth getting explicit here, something like "Grammatically incorrect misspelled messages written by you tell us more useful information than something generated by AI."
rcomer commentedon Dec 18, 2025
There are still valid uses of AI for writing the PR summary, e.g. using Google Translate if your English is not strong.
Also, how do we enforce the rule? Of course I often see text that I think looks AI generated, but I cannot prove it.
I am 👍 for limiting the number of PRs from new contributors.
timhoffm commentedon Dec 18, 2025
My words are only a rough idea, not necessary our final spelling. We can get more explicit, but the point is more general. "A well thought through solution and description with some rough language edges is better than a smoolth sounding text with logical errors."
Just a translation is ok. That should affect content and logic only in minor ways. In other words: Acceptable: "Please translate ...", Not-acceptable: "Summarize ..." / "Write a text ..."
We can decide on the degree of enforcement:
timhoffm commentedon Dec 18, 2025
We may want to try GitHub AI Issue and Comment Moderator. For a start only for information purposes.
16 remaining items
rcomer commentedon Feb 7, 2026
Pandas’ policy does explicitly require disclosure of AI use
https://pandas.pydata.org/docs/dev/development/contributing.html#automated-contributions-policy
melissawm commentedon Feb 7, 2026
I will cross-post here just for visibility: #31026 (comment)
rcomer commentedon Feb 8, 2026
This looks potentially interesting: a way for projects to learn from each other whose contributions they do and don’t trust ghostty-org/ghostty#10559
story645 commentedon Feb 8, 2026
Hmm wondering how this would intersect w GSOC applicants, given we're probably gonna be seeing PRs from folks who wanna apply (probably already have seen some) for the next month and a half (apps close in April). Could be useful for catching the folks spamming all the projects, but I think less useful on the trust side cause one of the goals of gsoc is bringing in folks who would be unlikely to have folks who could vouch for em.
rcomer commentedon Feb 8, 2026
Yeah, I was thinking more on the “don’t trust” side as someone putting up AI slop in one project is likely to be doing it elsewhere too..
Follow up from matplotlib#30848 discussion, added note in
rcomer commentedon Feb 9, 2026
Could we also add some advice along these lines to the policy? #31013 (comment)
I think it's an important distinction that we want a direct translation of what the person wrote, rather than something an LLM embellished with possibly spurious information.
Follow up from matplotlib#30848 discussion, added note in