TL;DR: It's okay to close a bug, if you can justify it properly.
Recently there has been a spate of bug-closing with what I would call
inadequate documentation. Comments such as "Obsolete?" or "I assume
it's fixed" could be applied to nearly every open bug we have. While
this does reduce the open bug count--something I have been watching
with morbid fascination for years--I do fear that the reduction is
potentially artificial, and incorrectly puts the onus on the original
bug author to reopen the case.
I suggest that closing a bug can be done IF AND ONLY IF you also state
one of the following:
- that revision NNNNNN actually fixed the bug
- that the bug cannot be reproduced with revision NNNNNN
- that the circumstances for the bug don't apply anymore; e.g.,
"This is about the makefiles and we don't use makefiles anymore."
- sound reasons for not fixing something (WONTFIX)
- some specific and plausible reason to think that a given bug is
otherwise inapplicable or obsolete
In particular, "Obsolete?" and "I assume it's fixed" are NOT enough
justification to close a bug.
If people are okay with this, I'd expect adding a new section to the
Developer Policy is probably the right place to put it.
Sounds reasonable to push back on these bug closings (for instance by replying to the bugs/reopening them and asking for clarification) - not sure it warrants a particularly documented policy, but I don’t much mind either way.
I did ping one, with no answer. There have been dozens in the past few days. I suppose I could have tried harder, but without something to point to it’s my opinion versus somebody else’s.
I think what Vitaly and others are doing is OK if we just tweak the messaging.* In these cases where the bug has been open for more than a year and there isn’t enough information to prove or disprove the existence of the bug, I think it’s OK to close the bug (probably as WORKSFORME), but invite the user to reopen with more information if the problem is still affecting them. Otherwise we have to go through a dance of pinging the bug, asking for more info, and then close it a month later in the likely case that the original reporter has moved on and cannot reproduce the problem. I think it’s better to communicate that, without more input, the community doesn’t plan to take action.
BTW, bugzilla’s messaging is totally crazy and user hostile to begin with. Users often have problems that are not compiler bugs, and we correctly close them as “INVALID”, all caps. It’s not a great experience.
P.S. How long has it been now since users have had to email llvm-admin to create bugzilla accounts? This is an issue, we need to find a way to lower the barrier for bug reporting.
I should have said up front that I am ecstatic to see somebody closing old probably-worthless bugs.
Better messaging is exactly what I’m after. We (properly) don’t tolerate “Fix bug” as a commit message; we shouldn’t tolerate similarly opaque bug-closing messages. It’s the same principle of proper project communication.
I’m fine with inviting the reporter to reopen if they still do care. What I’m not fine with is closing a bug for no clear reason, or at least no clearly stated reason.
I’m also not open to an argument along the lines of: I’ve closed 100 bugs this week and I’m sick of repeating myself, everybody has seen this message 100 times already, I don’t need to paste it in again.
The buried assumption there is that everyone is on llvm-bugs, which is patently not the case. The most important person to communicate with, every single time, is the reporter; and while the reporter has to be registered with Bugzilla, we should assume they are not on the bugs list.
TL;DR: It's okay to close a bug, if you can justify it properly.
Recently there has been a spate of bug-closing with what I would call
inadequate documentation. Comments such as "Obsolete?" or "I assume
it's fixed" could be applied to nearly every open bug we have. While
this does reduce the open bug count--something I have been watching
with morbid fascination for years--I do fear that the reduction is
potentially artificial, and incorrectly puts the onus on the original
bug author to reopen the case.
I suggest that closing a bug can be done IF AND ONLY IF you also state
one of the following:
- that revision NNNNNN actually fixed the bug
There is a field in bugzilla called "Fixed By Commits" that I added
specifically for this information.
I’m one of those that closed at least one bug in this manner, without citing exact DNNN nor rNNN numbers. I’d like to fix that and do better in the future.
So I agree 100% with you that we really ought to be more thorough and diligent with the way we’re handling bugs.
I for one would really like for the bug filing/tracking mechanisms to be easier/simpler/convenient/ergonomic. In particular:
- Let’s automate as much of this as possible. I really like the idea of auto-closing bugs with some convention based on the commit message(s).
- Let’s have a process/policy by which we handle un-acknowledged or mis-assigned bugs. A triage process would be really helpful, and I understand that a lot of us are volunteers and/or part-time working on things.
- Let’s evaluate whether bugzilla is still the appropriate solution for issue tracking. Considering that there’s (huge) efforts for re-licensing and the proposal to move to using git (and a monorepo structure?) then maybe alternatives can be explored for the next iteration of the development process when those are done — including whether LLVM should stick with Bugzilla or use GitHub issues (as one example) instead with a lot of the automation features already in place.
This is perhaps a longer conversation that needs to happen and I’m interested in having that conversation (it’s very relevant to my interests).
Most of bugs I closed this way are just “that the bug cannot be reproduced with trunk” or “not enough information and I don’t expect reporter is going to provide any after so may years”.
If I misinterpret the bug, I expect push back from reporter.
That's not unreasonable, especially if there hasn't been a concerted
effort to ensure all bugs have sufficient information previously.
Projects such as WINE introduce a 'NEEDSINFO' bug status, and
automatically close bugs that have had that status for a certain
length of time (in their case, 1 year)
<Bugs - WineHQ Wiki. We could imagine
adopting a similar process.
