Using clang for cross-compiling to Cortex M4

I'm having a hard time figuring out how to build clang/llvm to let me target a small Cortext M4 MCU. I've got HEAD clang installed, but apparently no runtime or standard libraries. I've read through

  http://clang.llvm.org/docs/CrossCompilation.html

and took a look at

  http://llvm.org/docs/CMake.html

It talks about how to build LLVM to target an architecture, but I thought LLVM was able to target all its targets without having to be built specifically for each target. The clang/llvm I have install has all targets enabled, and I can pass it -target arm-none-eabi -mcpu=cortext-m4, and it seems to do the right thing.

What I'd like to do is make a homebrew formula to build an LLVM-based toolchain to target small ARM MCUs (I realize I'll have to use GNU binutils to link, that's fine). But I can't really figure out how to build it even manually.

Hi Rick,

That's right, you're going to need ARM libraries. I know Compiler-RT
works well on ARMv7A, but I'm not sure about v7M, maybe Tim can help
you on that.

Another way is to have multilib gcc/lib packages installed on your
box, so that you can use GNU's libraries, as well as ld and as. The
integrated assembler is on by default on ARM on trunk (not on 3.4),
but if you encounter problems you can use -no-integrated-as option and
GNU as will be called instead.

Remember to add -I and -L paths to your compilation, as distributions
do vary greatly where they are...

Hope that helps,
--renato

Thanks, Renato.

Is compiler-rt sufficient? That doesn't provide the std libs, right?

I have a working installation of clang/llvm HEAD, I'm guessing I should use that to compile complier-rt, but I want to target ARMv7M...how do I build compiler-rt that way? I don't see it specified in either the docs for compiler-rt, or clang.

I checked out compiler-rt as shown here: http://compiler-rt.llvm.org

Are there options to pass to cmake?

Oh, nevermind. Even those basic compiler-rt build instructions don't work:

  https://pastee.org/h4f68

I give up.

Thanks. The thing is, does that build compiler-rt for a particular target, or for the host? Or does it build many, one for each target supported by LLVM?

Moreover, I want it to build compiler-rt using the very clang/llvm I checked out. Does it do compiler-rt using those freshly-built tools?

Do you intend to use C++ or just C?
I assume your target is Linux?

Definitely C++, but RTTI and exceptions aren't necessary (although, nice to have).

Do you have a working sysroot for your target system?
What is your host platform from the cross compiler?

*FOR the crosscompiler…the platform that runs the compiler.

The host is the latest released OS X. I do not have a working sysroot; that's what I'm trying to achieve. I thought compiler-rt was part of getting that into shape.

‘sysroot’ is not your toolchain. sysroot is the rough equivalent of an Apple SDK. That is where your system headers and libraries are located so that the compiler can find them. I have successfully built a mac/clang cross compiler for powerpc-unknown-linux. I got my sysroot out of the yocto/openembedded project build for my board. If you don’t have a sysroot, you can probably create one just by creating a directory on OSX that contains the contents of /usr/include and /usr/lib and maybe /lib.

What worked for me was to:

  1. build clang for OSX (host platform) as usual.
    a) configure it to install into a specific directory that will hold your tool chain.
    b) compile a simple program for OSX and make sure that works.

  2. setup your sysroot in some way.

  3. determine from clang the target triplet it requires. mine is 'powerpc-unknown-linux’

  4. get the source code for binutils. There are online tutorials for cross compiling binutils.
    a) configure it to target your platform as the default platform. If you don’t, it will assume OSX is the target and not build pieces like ‘ld’. use the triplet from #3
    b) when you configure, set the PREFIX value so that you don’t install it into your OSX system
    c) install it. you end up with a directory that has all the pieces staged.
    d) manually merge those pieces into the directory you have clang installed

  5. set up a simple C program that includes nothing. just have a main function that returns argc;

  6. get that to build. here is my redacted compile line to show you how to call your compiler:

“/linux_toolchain_root_3.8/bin/clang++" -target powerpc-unknown-linux -m32 -mcpu=e500v2 --sysroot=“/linux_sysroot_3.8” -I “/linux_sysroot_3.8/usr/include/c++” -I “/usr/include/c++/powerpc-fsl_networking-linux-gnuspe” -ggdb -Os -std=gnu++11 -MD -MP -pthread -c “…/…/Embedded/SysCommon/VSysLog.cc” -o “…/…/Embedded/SysCommon/VSysLog.o”

redacted linker line:

“/linux_toolchain_root_3.8/bin/clang++" main.o whatever.o … -lgcc_s -lstdc++ -lrt -lpthread -target powerpc-unknown-linux --sysroot=“/linux_sysroot_3.8" -stdlib=libstdc++ -std=gnu++11 -o SuperNodeDmxUpdater.elf

“” just means some arbitrary path. depends on how you want to structure your layout.

I don’t believe I’m using compiler-rt. I don’t see it in my toolchain or sysroot. I don’t recall doing anything special to build it. It was straightforward to set up clang and llvm. My strategy is to use my platform’s runtime rather than what comes with llvm & clang. The unwinder library doesn’t work on linux. that is why you see me using -lgcc_s. I believe that that component is supplied by libgcc in my setup. If I do a verbose link, I get…

“/linux_toolchain_root_3.8/bin/powerpc-unknown-linux-ld" —sysroot=/linux_sysroot_3.8 --eh-frame-hdr -m elf32ppclinux -dynamic-linker /lib/ld.so.1 -o myexe /linux_sysroot_3.8/usr/lib/crt1.o /linux_sysroot_3.8/usr/lib/crti.o /linux_sysroot_3.8/usr/lib/crtbegin.o -L/linux_sysroot_3.8/lib -L/linux_sysroot_3.8/usr/lib /main.o … -lgcc_s -lstdc++ -lrt -lpthread -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc linux_sysroot_3.8/usr/lib/crtend.o /linux_sysroot_3.8/usr/lib/crtn.o

  1. when you go to build a simple program, you’ll get errors telling you what is missing, you then have to find those missing elements and adjust your compile/linker lines to find them. In some cases, I added symlinks to my sysroot tree to map low-level runtime files like crt1.o to where my toolchain was expecting to find them.

  2. keep using trial and error until you can build a c++ program with dependencies.

  3. I did get libc++ to build using the libstdc++.so option and with the system’s unwinder (libgcc_s). I haven’t tested it out yet. I had a little trouble with libc++ because I don’t speak CMAKE. So, I had to unravel some of that to get all the compiler/linker flags right. When you get to that stage, let me know and I’ll give you my notes. I made it a point to take notes by the time I got to libc++.

Hopefully, this will give you enough info to muddle your way through it. it isn’t hard, but it can be hard to know where to go since there is no one authoritative list of steps. You have to look at all the tutorials and documentation and cherry-pick what you need as you go.

-James

Not really, RT is a libgcc replacement, not a standard library, and
C++ also needs libc++/libc++abi. Sorry I misled you, I thought you
were just looking for the compiler libraries.

For a quick working setup, having the GNU toolchain set up for your
target is the least trouble. You should be able to get a
cross-compiler ARM gcc with libraries and binutils on Mac, though I
don't know how to (I'm a Linux user myself).

cheers,
--renato

Is compiler-rt sufficient? That doesn't provide the std libs, right?

Not really, RT is a libgcc replacement, not a standard library, and
C++ also needs libc++/libc++abi. Sorry I misled you, I thought you
were just looking for the compiler libraries.

Yeah, I figured there was more to it.

For a quick working setup, having the GNU toolchain set up for your
target is the least trouble. You should be able to get a
cross-compiler ARM gcc with libraries and binutils on Mac, though I
don't know how to (I'm a Linux user myself).

Yeah, I'll probably do that, or try newlib.

Thanks, James, that gives me a great starting point.

Now to figure out how to write a homebrew formula to do this :wink:

I don’t know if that will be practical. this stuff requires a lot of hand-jiggering to get it right. But, It will be a learning exercise! :slight_smile:

What you COULD do that would be maintainable is to get Yocto working on OSX. Yocto is a package manager made for cross-compiling for embedded projects. The problem with it is the host-recipes aren’t OSX-friendly. They are all linux-specific. Yocto itself can be made to run on OSX with a few minor patches and putting certain gnu tools place of OSX tools (sed comes to mind). Once you have that Yocto would be able to drive your toolchain the same way it does on Linux.

The only other major complication to using OSX to cross compile for linux is the tendency by linux folk to make use of case-sensitivity in their filesystems. You have to put their stuff on special volumes and even then the Finder doesn’t entire work correctly with the files.

But, it can be done for those of us who like a more luxury unix environment.

-James

I don’t know if that will be practical. this stuff requires a lot of hand-jiggering to get it right. But, It will be a learning exercise! :slight_smile:

Well, any steps I can issue on the command line should be translatable into a recipe.

What you COULD do that would be maintainable is to get Yocto working on OSX. Yocto is a package manager made for cross-compiling for embedded projects. The problem with it is the host-recipes aren’t OSX-friendly. They are all linux-specific. Yocto itself can be made to run on OSX with a few minor patches and putting certain gnu tools place of OSX tools (sed comes to mind). Once you have that Yocto would be able to drive your toolchain the same way it does on Linux.

The only other major complication to using OSX to cross compile for linux is the tendency by linux folk to make use of case-sensitivity in their filesystems. You have to put their stuff on special volumes and even then the Finder doesn’t entire work correctly with the files.

Why? Why would anyone do that?

I don’t know if that will be practical. this stuff requires a lot of hand-jiggering to get it right. But, It will be a learning exercise! :slight_smile:

Well, any steps I can issue on the command line should be translatable into a recipe.

sure. but, is it going to work for all flavors of linux and whatever different ways there are of configuring binutils? over time as the projects evolve?

For instance, I determined experimentally where those c runtime binaries are stored in my sysroot and where clang/ld wanted to find them. Are they going to be in the same place on all flavors of linux across all targets?

If it has to be constantly updated, it would be pain to maintain. But, maybe it can be done. go for it. Its worth the education. I don’t know the answer. I did just enough to serve my purposes.

The only other major complication to using OSX to cross compile for linux is the tendency by linux folk to make use of case-sensitivity in their filesystems. You have to put their stuff on special volumes and even then the Finder doesn’t entire work correctly with the files.

Why? Why would anyone do that?

Linux folks work on Linux as mac folks work on macs. Its a well-defined feature on Linux that will always be there. People who aren’t interested in being on anything other than Linux will want to use that feature to their convenience. The Linux Kernel is an example of a project that requires a case-sensitive file system. We aliens must adapt to their turf if we want to use their stuff. :slight_smile:

Its not hard to work around…just a bit of an extra pain.

I don’t know if that will be practical. this stuff requires a lot of hand-jiggering to get it right. But, It will be a learning exercise! :slight_smile:

Well, any steps I can issue on the command line should be translatable into a recipe.

sure. but, is it going to work for all flavors of linux and whatever different ways there are of configuring binutils? over time as the projects evolve?

For instance, I determined experimentally where those c runtime binaries are stored in my sysroot and where clang/ld wanted to find them. Are they going to be in the same place on all flavors of linux across all targets?

If it has to be constantly updated, it would be pain to maintain. But, maybe it can be done. go for it. Its worth the education. I don’t know the answer. I did just enough to serve my purposes.

This is on OS X (homebrew is a package manager of sorts for OS X that works by building everything from source. The forumula are well-defined Ruby scripts). So, yes, I think, generally, it will be the same, at least for a given version of Clang/LLVM.

The only other major complication to using OSX to cross compile for linux is the tendency by linux folk to make use of case-sensitivity in their filesystems. You have to put their stuff on special volumes and even then the Finder doesn’t entire work correctly with the files.

Why? Why would anyone do that?

Linux folks work on Linux as mac folks work on macs. Its a well-defined feature on Linux that will always be there. People who aren’t interested in being on anything other than Linux will want to use that feature to their convenience. The Linux Kernel is an example of a project that requires a case-sensitive file system. We aliens must adapt to their turf if we want to use their stuff. :slight_smile:

Its not hard to work around…just a bit of an extra pain.

What I mean is, it's confusing for a human to look at two files that differ only in case. This is why Apple chose to go with a case-insensitive file system (it's case-preserving).

In any case, I think I'll skip Yocto for now. Your steps are still extremely applicable. Thanks!

I have just added C++ (see https://github.com/labapart/polymcu/blob/master/Application/Examples/BaremetalCpp/main.cpp) support to PolyMCU (https://github.com/labapart/polymcu).
PolyMCU is an Open Source firmware framework that is based on CMake that supports GCC and LLVM. It primarily targets ARM Cortex-M (even if I hope to add other architectures in the future). Because it is based on CMake you can build your firmware on Linux/Windows/MacOS.
It also uses Newlib and supports Baremetal/CMSIS RTOS (RTX)/FreeRTOS.

I also wrote a blog where I compared GCC and LLVM build size on ARM Cortex-M: http://labapart.com/blogs/3-the-importance-of-the-toolchain-version-in-embedded-space
Interesting results, Clang generated code is not much bigger than GCC on Cortex-M...