Transition C->bitcode->assembly->object looses frame pointers

Hi everyone,

We're doing some compile-time instrumentation according to the following scheme:

  llvm-gcc -O1 -g $in -emit-llvm -S -o $name.ll
  opt -O2 -load Instr.so $INSTR_FLAGS $name.ll -S -o $name-inst.ll
  llc $name-inst.ll -o $name.S
  g++ -c $name.S

However it turns out that the code instrumented this way is missing
frame pointers (e.g. backtrace() and/or libunwind cannot unwind the
stack when necessary).
Moreover, removing the instrumentation doesn't help: a simple
transition to and from bitcode doesn't help:

  llvm-gcc -O1 -g $in -emit-llvm -S -o $name.ll
  llc $name.ll -o $name.S
  g++ -c $name.S

We also tried to use Clang and to add -fno-omit-frame-pointers and
--disable-fp-elim -- do not work either.

On the other hand, compiling the code in a single pass of
gcc/llvm-gcc/clang does preserve the frame pointers.

Does anyone happen to know where the problem might be? Are there any
other flags we're missing?

Thanks,
Alexander Potapenko

Hi,

The -disable-fp-elim flag must be passed to the part of the compilation process that actually performs that optimisation pass - when you run clang/llvm-gcc in one invocation, obviously it will just be that invocation, but in your first case it would be the llc command. That is where the IR gets lowered, so is where the FP elimination takes place.

Pass flags are not, as far as I'm aware, propagated in generated bitcode files - the same flags need to be given to each part of the pipeline.

Hope this helps,

James

Hi James,

We've indeed passed the appropriate (and even excessive) flags to the
appropriate pipeline parts, that is:

llvm-gcc -O1 -fno-omit-frame-pointers -g $in -emit-llvm -S -o $name.ll
llc --disable-fp-elim $name.ll -o $name.S
g++ -fno-omit-frame-pointers -c $name.S

, but that didn't work

Alex

Below is an example of using backtrace() obtained from man backtrace
and the test results:

$ gcc bt.c -o bt-gcc
$ ./bt-gcc 4
backtrace() returned 4 addresses
./bt-gcc() [0x4007e3]
./bt-gcc() [0x400874]
./bt-gcc() [0x40089b]
./bt-gcc() [0x400894]

$ llvm-gcc bt.c -o bt-llvm-gcc
$ ./bt-llvm-gcc 4
backtrace() returned 4 addresses
./bt-llvm-gcc() [0x40074b]
./bt-llvm-gcc() [0x400809]
./bt-llvm-gcc() [0x400837]
./bt-llvm-gcc() [0x400830]

$ llvm-gcc -fno-omit-frame-pointer -g bt.c -emit-llvm -S -o bt.ll
$ llc --disable-fp-elim bt.ll -o bt.S
$ gcc -fno-omit-frame-pointer bt.S -o bt-llc
$ ./bt-llc 4
backtrace() returned 1 addresses
./bt-llc() [0x4007f0]

$ cat bt.c
       #include <execinfo.h>
       #include <stdio.h>
       #include <stdlib.h>
       #include <unistd.h>

       void
       myfunc3(void)
       {
           int j, nptrs;
       #define SIZE 100
           void *buffer[100];
           char **strings;

           nptrs = backtrace(buffer, SIZE);
           printf("backtrace() returned %d addresses\n", nptrs);

           /* The call backtrace_symbols_fd(buffer, nptrs, STDOUT_FILENO)
              would produce similar output to the following: */

           strings = backtrace_symbols(buffer, nptrs);
           if (strings == NULL) {
               perror("backtrace_symbols");
               exit(EXIT_FAILURE);
           }

           for (j = 0; j < nptrs; j++)
               printf("%s\n", strings[j]);

           free(strings);
       }

       static void /* "static" means don't export the symbol... */
       myfunc2(void)
       {
           myfunc3();
       }

       void
       myfunc(int ncalls)
       {
           if (ncalls > 1)
               myfunc(ncalls - 1);
           else
               myfunc2();
       }

       int
       main(int argc, char *argv[])
       {
           if (argc != 2) {
               fprintf(stderr, "%s num-calls\n", argv[0]);
               exit(EXIT_FAILURE);
           }

           myfunc(atoi(argv[1]));
           exit(EXIT_SUCCESS);
       }

Hi Alexander,

We're doing some compile-time instrumentation according to the following scheme:

   llvm-gcc -O1 -g $in -emit-llvm -S -o $name.ll
   opt -O2 -load Instr.so $INSTR_FLAGS $name.ll -S -o $name-inst.ll
   llc $name-inst.ll -o $name.S

llc eliminates frame pointers by default, see the -disable-fp-elim and
-disable-non-leaf-fp-elim options.

Ciao, Duncan.

Hi Alexander,

We've indeed passed the appropriate (and even excessive) flags to the
appropriate pipeline parts, that is:

  llvm-gcc -O1 -fno-omit-frame-pointers -g $in -emit-llvm -S -o $name.ll
  llc --disable-fp-elim $name.ll -o $name.S
  g++ -fno-omit-frame-pointers -c $name.S

, but that didn't work

it should work, so at this point I suggest you provide a concrete example
in which frame pointers are nonetheless eliminated.

Ciao, Duncan.

Hi Alexander, while I can reproduce the problem with your example,
inspection of the assembler shows that the frame pointer is being
correctly set up in each function. So I don't think the problem
is due to frame pointer elimination.

Ciao, Duncan.

Hi Alexander, I took a look at how backtrace works and it doesn't use
the frame pointer at all, instead it uses dwarf unwind tables. So to
make it work using llc you need to do as follows:

$ llc -unwind-tables bt.ll -o bt.S
$ gcc -o bt-llc bt.S
$ ./bt-llc 4
backtrace() returned 4 addresses
./bt-llc() [0x4007aa]
./bt-llc() [0x400856]
./bt-llc() [0x40087c]
./bt-llc() [0x400875]

Ciao, Duncan.

Hi Duncan,

I really had to look into backtrace internals before asking.
Thank you very much!

Alexander