Currently, it doesn’t seem like there is any documentation/restricted platforms for llvm-exegesis. There are a couple key points to make in regards to design that impact its current platform dependence:
llvm-exegesis relies on the Linux perf subsystem through libpfm. This means almost all useful functionality is gated behind a Linux dependency. With the recent addition of --dummy-perf-counters, execution of a snippet/generated snippet should work (although I haven’t tested it).
Certain architecture implementations might be restricted more than just to that architecture. For x86 specifically, there is currently no support for 32-bit x86. I’m thinking the situation is somewhat similar for ARM/PPC?
In regards to architectures that LLVM supports, it seems like there is general agreement about the architectures that llvm-exegesis supports. X86 (64-bit only), some ARM (seems like only AArch64), some PPC, etc. It would probably be good to document what architectures are supported somewhere however.
I’m bringing this up mainly because of an RFC ([RFC] Memory annotations for llvm-exegesis) I posted earlier and I’m trying to figure out what platforms need to be supported if we make it the default method of execution rather than opt-in.
Supported platforms is harder to decipher from the code base. There is a very reduced set of functionality on operating systems other than Linux, but there is still support for some features on Windows, and this functionality seems to be somewhat maintained (e.g., ⚙ D147926 [llvm-exegesis] Fix ifdefs in X86/Target.cpp).
So my main points/questions are the following:
We should probably decide on and document the architectures and platforms that we plan on supporting.
Is anyone using llvm-exegesis on Windows? If not, I’d like to propose stating that we only support Linux to make development easier.
Thanks for the feedback! Having a README.md in /llvm/tools/llvm-exegesis would probably be a good idea anyways ([Proposal] Encourage README.md files for sub-directories), and it would make sense to note what platforms we support and why (all the developer facing issues) there. A short blurb in the user-facing documentation would definitely be good as well as since these decisions do impact users.
My main point with this post is trying to figure out what platforms people are currently using llvm-exegesis on. I’m reasonably confident than no one is using outside of Linux due to the issues mentioned in the first post, but given recent patches, particularly from @kpdev42, I’m not so sure.
I’ll open up a patch adding a README.md with the proposed information along with a blurb in the user-facing documentation soon (probably in a couple days) taking feedback from here and also pinging people collaborating on llvm-exegesis in tree in case they haven’t responded here.
Thank you for bringing this topic. I’m not sure that Windows-specific code was thoroughly tested except for buildability (especially because __x86_64__ and _MSC_VER defines were used together). Before ⚙ D146301 [llvm-exegesis] Make possible to execute snippets without perf counters llvm-exegesis was probably only tested on Linux. Because now running snippets on Windows is at least technically possible (with --use-dummy-perf-counters option no libpfm and perf_event_open support required), ⚙ D147926 [llvm-exegesis] Fix ifdefs in X86/Target.cpp was introduced as it can help in basically checking llvm-exegesis in more development environments (but strictly speaking, the benchmark code may crash or not depending on the native ABI). The ⚙ D147926 [llvm-exegesis] Fix ifdefs in X86/Target.cpp was mostly implemented because the tests in ⚙ D146301 [llvm-exegesis] Make possible to execute snippets without perf counters failed on X86_64 Windows, so options were either explicitly state that llvm-exegesis is unsupported on Windows or fix it - the decision to choose the latter was mostly arbitrary because of absense of information.
Considering what platforms are currently supported, it may be worth trying to run a command like this:
and look if error message is printed stating that the target is completely unsupported. If target is supported, there are still high chances that exegesis cannot generate a snippet for this particular combination of options due to some target-specific reasons.
Thank for you for the additional information. I reached similar conclusions in regards to testing on other platforms. I still don’t think there’s a lot of coverage for the actual snippet execution. Thanks for bringing up the ABI differences! That’s definitely another factor to consider in regards to platform independence.
Seems like we’re running into similar things here in regards to a lack of clarity on where llvm-exegesis is supported. Thanks for the suggestion for testing platforms by specifying the triple. I’ll look into that when I go to write some of the documentation.
Given that no one seems to be using llvm-exegesis on other platforms (particularly Windows) in addition to all the platform dependence of llvm-exegesis, I think it’s reasonable to explicitly say that we only support Linux, at least at this time. I’ll post a documentation patch and then look at seeing if we could remove any complexity after this change (assuming everyone is agreement), but I don’t think there will be any gains there.