Abnormal behavior on M1 MacOS


I’m a software developer. I build open-source projects in my spare time. One of them uses LLVM as a third party dependency.
I recently noticed a few issues in llvm on M1 chip.
These issues exist probably because M1 is still a relatively new thing. But I would like to bring these issues on the table so that you are aware of them.

First of all, I have to run the build in Rossetta compatible terminal because of this error. If you build llvm on native terminal without Rosetta support, it ends here.

/bin/sh: line 1: 20715 Killed: 9 …/…/…/bin/llvm-tblgen -gen-attrs -I /Users/caojiayin/Desktop/GitHub/llvm-project-master/llvm/include/llvm/IR -I/Users/caojiayin/Desktop/GitHub/llvm-project-master/build_new/include -I/Users/caojiayin/Desktop/GitHub/llvm-project-master/llvm/include /Users/caojiayin/Desktop/GitHub/llvm-project-master/llvm/include/llvm/IR/Attributes.td --write-if-changed -o /Users/caojiayin/Desktop/GitHub/llvm-project-master/build_new/include/llvm/IR/Attributes.inc

/bin/sh: line 1: 20713 Killed: 9 …/…/…/bin/llvm-tblgen --gen-directive-impl -I /Users/caojiayin/Desktop/GitHub/llvm-project-master/llvm/lib/Frontend/OpenACC -I/Users/caojiayin/Desktop/GitHub/llvm-project-master/build_new/include -I/Users/caojiayin/Desktop/GitHub/llvm-project-master/llvm/include /Users/caojiayin/Desktop/GitHub/llvm-project-master/llvm/include/llvm/Frontend/OpenACC/ACC.td --write-if-changed -o /Users/caojiayin/Desktop/GitHub/llvm-project-master/build_new/lib/Frontend/OpenACC/ACC.cpp

make[2]: *** [include/llvm/IR/Attributes.inc] Error 137

make[2]: *** [lib/Frontend/OpenACC/ACC.cpp] Error 137

make[1]: *** [include/llvm/IR/CMakeFiles/intrinsics_gen.dir/all] Error 2

make[1]: *** Waiting for unfinished jobs…

make[1]: *** [lib/Frontend/OpenACC/CMakeFiles/acc_cpp.dir/all] Error 2

According to LLVM documents, ‘LLVM_TARGET_ARCH:STRING’ can be set either to the target arch or host so that the arch of the host will be picked.
This does work as expected on Intel MacOS, but the same doesn’t go true for M1 Mac. The issue manifests itself as crashing during run-time once the builds are generated. For example, there is this example program called ‘HowToUseLLJIT’, this will result in this error if that macro is set to host.

./bin/HowToUseLLJIT: Unable to find target for this triple (no targets are registered)

And if it is set to AArch64 explicitly, this will end up with this error

No available targets are compatible with triple "x86_64-apple-darwin20.2.0"

This error is the second issue I will talk about later in this email. Clearly, changing that ‘LLVM_TARGET_ARCH:STRING’ will result in different builds. Maybe CMake script doesn’t parse that macro on M1 correctly somehow. I’m not an expert of LLVM, but I hope this could be helpful enough for you guys to reproduce the problem and eventually fix it.

The second problem is the compatible issue that the example build emits.
To workaround the problem, I set the triple in code explicitly to 'arm64-apple-darwin20.2.0". And it works after the workaround. Somehow that triple is not correctly set inside LLVM. I also happen to notice this macro in CMake file cache ‘LLVM_DEFAULT_TARGET_TRIPLE’ is x86_64-apple-darwin20.2.0, which might be related.
This sounds like a bug in LLVM not correctly detect arch under the hood.

I tried finding resources on internet about how to setup LLVM on M1 Mac and there isn’t much.
So the whole setup process is a bit difficult. It would be nice to have some documents in the future.