Suggestions to debug a forever running clang compile command

Hi,
I’m wondering if there are general best practices to debug a clang compile command that takes forever?

I’m imagining clang options that print LLVM_DEBUG logs / warnings;

also I came across -ftime-report -mllvm -time-passes but it didn’t work for the particular case, since the command takes forever and won’t stop to give a report, and the existence a timeout option may not be well defined (compared with LLVM_DEBUG logs).

Any related experience will be helpful!

In similar situations, I’ve been using a profiler and killing the process after a minute to see where the time is spent, or using -mllvm --print-after-all to see the progress through the optimizer.

thanks Mehdi!

Both ideas sound very actionable, and using a profiler makes a lot of sense!

Not related to debugging, but just fwiw, D120295 fixes the specific clang command hangs.

If you happen to be using a Mac, spindump can be useful in situations like this.

1 Like

Seems like this has been solved but for the record - you could narrow down the scope by using your cc1 command and running it with -fsyntax-only which would run only the frontend actions and might tell you if the issue is in clang frontend or LLVM.

2 Likes

I’ve recently gone through a similar debugging flow - this is what I did (the following assumes you have a debug build of LLVM to work with).

  1. Determine if the long-running compile happens post-clang. i.e. see if adding -emit-llvm -S to your compile command results in a .ll file or just hangs. If it hangs, you’d need to use c-reduce to get a minimal test case. That wasn’t necessary for me, but I imagine a similar approach to the following would still work.
  2. Let’s assume the issue was in the middle or backend and you have a huge .ll file produced by clang. You’ll want to create as small a test case as possible. Use bugpoint or llvm-reduce for this (I’m not sure what the limitations/advantages of bugpoint vs llvm-reduce are at this point I’m afraid). You’ll want to set up a custom compile-commnd that fails after a certain time, picking a timeout that you’re pretty sure is long enough for the input to compile successfully on your machine if there wasn’t an infinite loop somewhere. In my case:
$ cat test.sh
#!/bin/sh
timeout 5 ./bin/llc "$@"
  1. Make sure you remember to chmod +x test.sh, and check it works as intended with ./test.sh testcase.ll; echo $? (which should indicate a non-zero return code after the timeout completes). Now invoke bugpoint: ./bin/bugpoint -compile-custom -compile-command=./test.sh testcase.ll
  2. Bugpoint will run for a while, and ultimately tell you it’s produced a file bugpoint-reduced-simplified.bc. Just run llvm-dis bugpoint-reduced-simplified.bc to get the .ll.
  3. Now you hopefully have a relatively short example, you can run it in llc with -debug -print-after-all and hopefully the loop is clear based on that.

Note that you could skip straight to the last step and not try to minimise the test case initially, but if your input is large it might take a very long time to reach the loop.

4 Likes

the option to run front-end only is great to know, and will be very helpful!