we have a great community of users and testers who take time to file bugs for various issues like crashes and miscompiles. That’s a valuable source of information for the project, but I think we aren’t making the most out of it.
Unfortunately it can take quite a long time from filing an issue to it being acknowledged and fixed. For example, LLVM/Clang contains a number of known miscompiles that can stay in the codebase for years.
Over the last couple of weeks I tried to work through many reported miscompiles and crashes to bring the information in the issue tracker more up-to-date and I am sharing his post to hopefully get more people interested. While working on fixing bugs might not be as exciting as working on new features, fixing some of these bugs will have a big positive impact on our users’ experience. If you are interested in helping out, there are multiple things you could do:
Update information in the issue tracker:
(One challenge is that we have a number of bugs that have been reported years ago, but never had any further comments.)
Try to reproduce reported issues,
tag them with appropriate labels,
possible attach a minimal IR reproducer.
Pick one or more issues and try to fix them.
Keep testing and filing issue for LLVM, we will eventually get to them
To wrap up, here are links to the lists of known miscompiles & crashes in Clang/LLVM, with the number of open bugs as of today. I may share infrequent updates as we hopefully make progress on squashing some of those!
Thanks for this @fhahn ! The LLVM community has historically closed about 2/3 of new reports, which has been remarkably stable over time. Flattening that line would be awesome. Maybe a project for when I retire
I’d put a picture of the data I have, but I don’t see a way to do that.
From my experience, we as a project are quite good at resolving miscompiles & crashes that can be tracked down to recent commits. Those usually get closed quite promptly.
Some of the issues in the shared lists might also be good candidate issues for new contributors, but it is always a bit difficult to judge unfortunately.
That would be great, I think the web-interface should allow uploading pictures. Not sure about the email integration though.
It looks like Github’s API provides access to all issues and there are tools to export all issues for offline analysis. I’ve not spent much time on that yet (the first script I tried wasn’t able to cope with the large volume of issues), but it might be interesting to get some more statistics. Perhaps someone has experience with that already?
That sounds great! I can’t really comment on which of the Clang crashes are more or less difficult to fix, but I’d recommend picking one, try to reproduce it, reduce the input and use a debugger to try to understand the issue.
I also just realized that discourse somehow mangled up the links to the issue queries somehow so that the filters won’t get applied
Since the initial post, a number of crashes and mis-compiles have been fixed. Thanks to everybody who helped out!
The current number of open mis-compiles and crashes increased since the original post, thanks to our users filing more issues and progress on tagging existing issue properly. Here’s the current snapshot:
