How to generate .bc file using configure && make on Mac OS X?

Hi,

I use the following commands to compile a GNU package to .bc files on Linux.

./configure CC=clang RANLIB=llvm-ranlib CFLAGS=-flto
LDFLAGS=-Wl\,-plugin-opt=save-temps\ -flto\ -fuse-ld=gold
make

But since it uses ld.gold, it won't work on Mac OS X. Is there
something equivalent that works on Mac OS X? Thanks.

Hi Peng,

On macOS you just omit the '-fuse-ld=gold' and it will work out of the box.

Cheers,
Alex.

On macOS you just omit the '-fuse-ld=gold' and it will work out of the box.

> ./configure CC=clang RANLIB=llvm-ranlib CFLAGS=-flto
> LDFLAGS=-Wl\,-plugin-opt=save-temps\ -flto\ -fuse-ld=gold
> make

No. It doesn't work on Mac OS by just omitting '-fuse-ld=gold'.

$ ../../../extract/bash-5.0/configure CC=clang CXX=clang++
RANLIB=llvm-ranlib CFLAGS=-g\ -flto
LDFLAGS=-Wl\,-plugin-opt=save-temps\ -flto
checking build system type... x86_64-apple-darwin17.7.0
checking host system type... x86_64-apple-darwin17.7.0

Beginning configuration for bash-5.0-release for x86_64-apple-darwin17.7.0

checking for gcc... clang
checking whether the C compiler works... no
configure: error: in `/Users/pengy/homebake/work/bash/build/Linux/y':
configure: error: C compiler cannot create executables
See `config.log' for more details

Here is the relevant content from config.log

The macOS linker supports -save-temps directly (but not -plugin-opt).
I haven't tried, but I'd guess that if you modified the -Wl part of
your command line in the obvious manner it'll work.

Cheers.

Tim.

I can run the configure using this command.

../../../extract/bash-5.0/configure CC=clang CXX=clang++
RANLIB=llvm-ranlib CFLAGS=-g\ -flto LDFLAGS=-save-temps\ -flto

But the command to generate the executable "bash" (called by make)
does not produce a .bc file.

clang -L./builtins -L./lib/readline -L./lib/readline -L./lib/glob
-L./lib/tilde -L./lib/sh -save-temps -flto -g -flto
-Wno-parentheses -Wno-format-security -o bash shell.o eval.o y.tab.o
general.o make_cmd.o print_cmd.o dispose_cmd.o execute_cmd.o
variables.o copy_cmd.o error.o expr.o flags.o jobs.o subst.o hashcmd.o
hashlib.o mailcheck.o trap.o input.o unwind_prot.o pathexp.o sig.o
test.o version.o alias.o array.o arrayfunc.o assoc.o braces.o
bracecomp.o bashhist.o bashline.o list.o stringlib.o locale.o
findcmd.o redir.o pcomplete.o pcomplib.o syntax.o xmalloc.o
-lbuiltins -lglob -lsh -lreadline -lhistory -ltermcap -ltilde
lib/intl/libintl.a -liconv -liconv -ldl

It is just the .o files are bitcode. But bash is an executable. And
there are not .bc files accompanying it.

$ file bash
bash: Mach-O 64-bit executable x86_64

Using the original commands on Linux, I got the following associated files.
./configure CC=clang RANLIB=llvm-ranlib CFLAGS=-flto
LDFLAGS=-Wl\,-plugin-opt=save-temps\ -flto\ -fuse-ld=gold
make

$ ls -gltr bash.*.bc
-rw-r--r-- 1 staff 4769876 Jan 25 10:16 bash.0.0.preopt.bc
-rw-r--r-- 1 staff 4769876 Jan 25 10:16 bash.0.2.internalize.bc
-rw-r--r-- 1 staff 4739688 Jan 25 10:16 bash.0.4.opt.bc
-rw-r--r-- 1 staff 4739688 Jan 25 10:16 bash.0.5.precodegen.bc

Peng,

You can also try using the gllvm wrapper: https://github.com/SRI-CSL/gllvm ; it’s quite reliable and easy to use in my experience.

-JK

This works great.

You can also try using the gllvm wrapper: https://github.com/SRI-CSL/gllvm ; it's quite reliable and easy to use in my experience.

Just to be sure. Only one version of bc file will be generated that
corresponds to the compiler options provide at configure?

The original approach using ld.gold will provide several .bc files
that correspond to different stages of the optimization.

I think using gllvm, I need to specify the corresponding -O? option to
the correct optimization stage of the bc file? Thanks.

As far as I understand, gllvm doesn’t run LTO pipeline or any cross-module optimization, and the optimization level provided is only used to compile each Translation Unit separately.
If you want to have LTO-like build, you can first run gllvm with -O3 -Xclang -disable-llvm-optzns (builds code with no optimizations, but doesn’t emit optnone and nounwind attributes) and later run opt on the final bitcode. Note that the results might not be the same as running a proper LTO build because of different pass manager pipelines.

but doesn’t emit optnone and nounwind attributes

s/nounwind/noinline

It seems that it does not really work. When I run the generated .bc
file, it gives me the following error. Do you what is wrong?

$ lli bash.bc
Stack dump:
0. Program arguments: lli bash.bc
0 lli 0x0000000105153922
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 37
1 lli 0x0000000105153d26 SignalHandler(int) + 200
2 libsystem_platform.dylib 0x00007fff7d6daf5a _sigtramp + 26
3 libsystem_platform.dylib 0x00007ffeeb06cb20 _sigtramp + 1838750688
4 lli 0x0000000104ed5045
llvm::MCJIT::runFunction(llvm::Function*,
llvm::ArrayRef<llvm::GenericValue>) + 535
5 lli 0x0000000104e67ae7
llvm::ExecutionEngine::runFunctionAsMain(llvm::Function*,
std::__1::vector<std::__1::basic_string<char,
std::__1::char_traits<char>, std::__1::allocator<char> >,
std::__1::allocator<std::__1::basic_string<char,
std::__1::char_traits<char>, std::__1::allocator<char> > > > const&,
char const* const*) + 1159
6 lli 0x0000000104b94b84 main + 9104
7 libdyld.dylib 0x00007fff7d3cc015 start + 1
Segmentation fault: 11

I did a sanity check and run some of the bitcode I complied with gllvm and it seems to work fine under lli. The only issues is to make sure you load all the (external) libraries the bitcode needs. I highly suggest you run a debug version of lli under a debugger and see what exactly happens.

Could you try the bash source code to see if you see the same errors?
It should just take a few minutes to configure/compile. Thanks.

https://ftp.gnu.org/gnu/bash/bash-5.0.tar.gz

It did work for me with llvm-5.0. I don’t have time run it on newer versions.

wget [https://ftp.gnu.org/gnu/bash/bash-5.0.tar.gz](https://ftp.gnu.org/gnu/bash/bash-5.0.tar.gz)
tar xvf bash-5.0.tar.gz
cd bash-5.0
mkdir build; cd build
export LLVM_COMPILER_PATH=my_llvm_dir/bin
export CC=gclang
export CXX=gclang++
../configure
make -j32
for i in $(find . -executable -and -type f) ; do echo $i ; extract-bc $i ; done
lli bash.bc -c "echo 'test''"

Here are some of my environment.

$ which clang
/Library/Developer/CommandLineTools/usr/bin/clang
$ clang --version
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
$ which lli
/usr/local/opt/llvm/bin/lli
$ /usr/local/opt/llvm/bin/clang --version
clang version 7.0.1 (tags/RELEASE_701/final)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin

I rerun the exact commands that you used. But I still get the error.

Does setting LLVM_COMPILER_PATH ensure the correct version of compiler
be used? Thanks.

export LLVM_COMPILER_PATH=/usr/local/opt/llvm/bin
export CC=gclang
../bash-5.0/configure && make -j2
for i in $(find . -executable -and -type f) ; do echo $i ; extract-bc $i ; done
$ lli bash.bc -c "echo 'test''"
Stack dump:
0. Program arguments: lli bash.bc -c echo 'test''
0 lli 0x0000000102808922
llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 37
1 lli 0x0000000102808d26 SignalHandler(int) + 200
2 libsystem_platform.dylib 0x00007fff7d6daf5a _sigtramp + 26
3 libsystem_platform.dylib 0x00000001037c9080 _sigtramp + 2249122112
4 lli 0x000000010258a045
llvm::MCJIT::runFunction(llvm::Function*,
llvm::ArrayRef<llvm::GenericValue>) + 535
5 lli 0x000000010251cae7
llvm::ExecutionEngine::runFunctionAsMain(llvm::Function*,
std::__1::vector<std::__1::basic_string<char,
std::__1::char_traits<char>, std::__1::allocator<char> >,
std::__1::allocator<std::__1::basic_string<char,
std::__1::char_traits<char>, std::__1::allocator<char> > > > const&,
char const* const*) + 1159
6 lli 0x0000000102249b84 main + 9104
7 libdyld.dylib 0x00007fff7d3cc015 start + 1
8 libdyld.dylib 0x0000000000000004 start + 2193833968
Segmentation fault: 11