Consumer ARM platform suitable for LLVM development?

Now that I have HLVM up and running on x86 and x64 I am interested in testing
it out on an ARM in preparation for their imminent world domination. I
believe LLVM already has excellent support for the ARM thanks to Apple's
ARM-based products. However, I'd really like an ARM-based consumer device
suitable for developing and testing HLVM on, rather than using a cross
compiler and uploading to an embedded device like a phone or lab equipment
like a beagle board.

Has anyone tried LLVM on one of the ARM-based netbooks like a Pandora, Touch
Book or maybe some pre-release Apple swankiness?

Before you commit to a specific hardware device, you can also prototype with an ARM emulator, such as Qemu.

The problem I’ve had is building an LLVM cross-compiler from Linux/x86 to Linux/ARM (as has another llvm-dev poster). Someone mentioned to me off-list that he managed to get it to build, but I haven’t been able to reproduce the build using his instructions (I’ll post my results in another thread).

Building LLVM inside Qemu is rather slow and painful – I’ve tried it once and didn’t get far with it. However, since LLVM + LLVM-GCC are shipped as part of the iPhone SDK, at least Darwin/x86 → Darwin/ARM cross is possible.

Misha

I've been successfully experimenting with an arm-softfloat-linux-gnu cross-compiler at the time of llvm 2.1. The easiest way to get this working is to use Dan Kegel's crosstool to build a regular gcc toolchain. This will give you binutils, libgcc, and glibc, which you will need anyway.

Building llvm-gcc was relatively painless. The only problem I've encountered was that my target had no hardware floating point support and the endianess for the softfloat implementation (libgcc) differed from integer endianess, which required some patches at the llvm side. I have not checked if this is still necessary. If you run into the same problems, I would be happy to share the patch (which I have updated to llvm 2.4 at some point).

The problem I’ve had is building an LLVM cross-compiler from Linux/
x86 to Linux/ARM (as has another llvm-dev poster). Someone
mentioned to me off-list that he managed to get it to build, but I
haven’t been able to reproduce the build using his instructions
(I’ll post my results in another thread).

I’ve been successfully experimenting with an arm-softfloat-linux-gnu
cross-compiler at the time of llvm 2.1. The easiest way to get this
working is to use Dan Kegel’s crosstool to build a regular gcc
toolchain. This will give you binutils, libgcc, and glibc, which you
will need anyway.

I’ve been trying to build for the target arm-none-linux-gnueabi, using the appropriate pre-built CodeSourcery toolchain, hence avoiding building my own gcc crosstool prior to starting building LLVM and LLVM-GCC. LLVM builds just fine, it’s building llvm-gcc that gives me this error:

gcc -c -g -O2 -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -DHAVE_CONFIG_H -DGENERATOR_FILE -I. -Ibuild -I…/…/…/…/src/llvm-gcc4.2-2.5.source/gcc -I…/…/…/…/src/llvm-gcc4.2-2.5.source/gcc/build -I…/…/…/…/src/llvm-gcc4.2-2.5.source/gcc/…/include -I…/…/…/…/src/llvm-gcc4.2-2.5.source/gcc/…/libcpp/include -I…/…/…/…/src/llvm-gcc4.2-2.5.source/gcc/…/libdecnumber -I…/libdecnumber -I/tmp/llvm-project/install/x86_64-arm/llvm/include -I/tmp/llvm-project/src/llvm-2.5/include -DENABLE_LLVM -I/tmp/llvm-project/install/x86_64-arm/llvm/include -D_DEBUG -D_GNU_SOURCE -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -o build/gencondmd.o build/gencondmd.c

…/…/…/…/src/llvm-gcc4.2-2.5.source/gcc/config/arm/arm.md:4788: error: ‘MACHO_DYNAMIC_NO_PIC_P’ undeclared here (not in a function)

This tells me there are some Darwin-specific bits in arm.md which shouldn’t be there (MachO is Mac OS X-specific). I’m using the attached script llvm-arm-crosstool.sh to configure/build/install LLVM and LLVM-GCC from the 2.5 release. If anyone wants to try it out and see if it works for them, I’d love to hear their experiences.

If any ARM/GCC experts know how to fix arm.md to not refer to Darwin-specific macros, that would be great, too.

Building llvm-gcc was relatively painless. The only problem I’ve
encountered was that my target had no hardware floating point support
and the endianess for the softfloat implementation (libgcc) differed
from integer endianess, which required some patches at the llvm side.
I have not checked if this is still necessary. If you run into the
same problems, I would be happy to share the patch (which I have
updated to llvm 2.4 at some point).

I would be interested in seeing your patch, if you wouldn’t mind sharing it. Bonus points if you have a script similar to mine above to simplify the entire process, or your notes/directions that I can convert into an easy-to-run script. :slight_smile:

Thanks,
Misha

llvm-arm-crosstool.sh (4.76 KB)

Probably the right general idea is to #define MACHO_DYNAMIC_NO_PIC_P to be 0 for non-Darwin targets. Not sure where to put this so it will work for both targets (the Darwin definition comes from config/darwin.h).

If any ARM/GCC experts know how to fix arm.md to not refer to Darwin-specific macros, that would be great, too.

Probably the right general idea is to #define MACHO_DYNAMIC_NO_PIC_P to be 0 for non-Darwin targets. Not sure where to put this so it will work for both targets (the Darwin definition comes from config/darwin.h).

I don’t think so. Darwin-specific pieces of code should be guarded, as usual, with TARGET_MACHO define.

We absolutely need someone to drive this. Some of us who work on the ARM target are developing and testing our work on proprietary platforms. We can’t share our infrastructure with the public. It would be nice if someone can figure out the following and document them:

  1. How to build llvm-gcc with ARM for various platforms?
  2. How to use llvm-gcc to run llvm test suite on qemu, or some other ARM based netbooks.
  3. Anything else of interest. :slight_smile:

Would someone step up and drive this please?

Thanks,

Evan

Perhaps, but the problem is that MACHO_DYNAMIC_NO_PIC_P is so ingrained into the machine description, that if you try to guard each use with something like

#ifdef TARGET_MACHO

#endif

you will get unreadable code for something as simple as:

; APPLE LOCAL begin ARM pic support
(define_insn “pic_load_addr_arm”
[(set (match_operand:SI 0 “s_register_operand” “=r”)
(unspec:SI [(match_operand:SI 1 “” “mX”)
(label_ref (match_operand 2 “” “”))] UNSPEC_PIC_SYM))
(use (label_ref (match_dup 2)))]
“TARGET_ARM && (flag_pic || (TARGET_MACHO && MACHO_DYNAMIC_NO_PIC_P))”
“ldr%?\t%0, %1”
[(set_attr “type” “load1”)
(set (attr “pool_range”) (const_int 4096))
(set (attr “neg_pool_range”) (const_int 4084))]
)

or, even more complex code sequence:

;; APPLE LOCAL begin compact switch tables
;; This pattern represents the library call for Thumb switch tables.
;; The functions’ (sparse) register usage is recorded as clobbers.

(define_insn “thumb_casesi_internal”
[(parallel [(set (pc)
(if_then_else
(leu (match_operand:SI 0 “s_register_operand” “l”)
(match_operand:SI 1 “const_int_operand” “i”))
(mem:SI (plus:SI (mult:SI (match_dup 0) (const_int 2))
(label_ref (match_operand 2 “” “”))))
(label_ref (match_operand 3 “” “”))))
(clobber (reg:CC CC_REGNUM))
(clobber (reg:SI LR_REGNUM))
(clobber (reg:SI IP_REGNUM))
(use (reg:SI 0))
(use (label_ref (match_dup 2)))])]
“TARGET_COMPACT_SWITCH_TABLES”
"*
{
rtx body = PATTERN (next_real_insn (insn));
static char buf[255];
gcc_assert (GET_CODE (body) == ADDR_DIFF_VEC);
strcpy(buf, "bl\t");
if (flag_pic || MACHO_DYNAMIC_NO_PIC_P)
strcat(buf, "L");
if (GET_MODE (body) == QImode
&& ADDR_DIFF_VEC_FLAGS (body).offset_unsigned)
{
register_switchu8_libfunc ();

[… more code below …]

If this code is purely Darwin-specific, it shouldn’t even appear in arm.md . If this is ARM-PIC-specific, then we should use a var such as ARM_PIC, and darwin.h can set the common ARM_PIC variable if MACHO_DYNAMIC_* is defined.

Adding #ifdef/#endif code to .md files will make what’s already barely readable even less comprehensible.

Misha

Attached is the patch I've been building arm-eabi with, which might
help with linux-gnueabi. I disable multilib to get around several bugs
with thumb. I build cross binutils first, then llvm, then llvm-gcc
with newlib merged in.

The following bugs need to be addressed for these non-Darwin ARM targets:
1388
2313
2545
2985

deep

fix_apples_bugs.diff (3.4 KB)

I've been trying to build for the target arm-none-linux-gnueabi, using the appropriate pre-built CodeSourcery toolchain, hence avoiding building my own gcc crosstool prior to starting building LLVM and LLVM-GCC. LLVM builds just fine, it's building llvm-gcc that gives me this error:
[...]
../../../../src/llvm-gcc4.2-2.5.source/gcc/config/arm/arm.md:4788: error: ‘MACHO_DYNAMIC_NO_PIC_P’ undeclared here (not in a function)

I think this error is a result of the switch to Appel's gcc 4.2. I cannot say if it's hard to fix these issues (I guess not), but if you need a quick solution, you might sill use the 4.0 based frontend. As far as I know, the bitcode format should be compatible to the current tools (somebody please correct me if I'm wrong).

Building llvm-gcc was relatively painless. The only problem I've
encountered was that my target had no hardware floating point support
and the endianess for the softfloat implementation (libgcc) differed
from integer endianess, which required some patches at the llvm side.
I have not checked if this is still necessary. If you run into the
same problems, I would be happy to share the patch (which I have
updated to llvm 2.4 at some point).

I would be interested in seeing your patch, if you wouldn't mind sharing it. Bonus points if you have a script similar to mine above to simplify the entire process, or your notes/directions that I can convert into an easy-to-run script. :slight_smile:

Sorry, there's no such script. But here's how I configured llvm-gcc:
configure --prefix=<wherever> --program-prefix=llvm-arm- --enable-llvm=<path to llvm> --enable-checking --target=arm-softfloat-linux-gnu --disable-shared --disable-nls --disable-threads --with-float=soft --enable-languages=c

You can find a tiny patch for llvm-gcc and the necessary changes for llvm 2.4 attached. The llvm patch should cleanly apply to llvm 2.4, but I had to pick the changes from a private repository with some modifications. Thus, the patch is untested and I cannot guarantee if I got all the necessary bits. Feel free to contact me if you run into troubles.

The patch allows to have different endianess for integer and floating point types. I'm not sure if this is the best way to handle this issue, but at least it seems to be working (I've used it to cross-compile SPECINT 2000). Note that you probably won't need all this if you have a hardware FPU.

Best,

llvm-gcc.patch (2.01 KB)

llvm2.4-softfloat.patch (10.4 KB)

Sandeep, many thanks! I was able to build a cross-compiler using your patch (with a minor change) and ran “hello world” in Qemu-ARM after compiling and linking it on x86_64. I put the list of outstanding bugs on a wiki page for tracking and reference: http://google2.osuosl.org/wiki/index.php/HowTo:_LLVM-based_cross_compiler

I’ll try to get the patch committed to llvm-gcc piece-meal. It looks like part of the patch includes the fix for http://llvm.org/PR2545 – Chris said he’s OK with the patch being committed, is there any reason it hasn’t been applied yet? Does anyone mind if I do?

Misha

What change did you need?

deep

Even with your change, it was still complaining about not having a definition of MACHO_DYNAMIC_NO_PIC_P somewhere, so I took the easy way out with inserting this in arm.h:

/* Overridden by arm/darwin.h, whether it is included first or not. */
#ifndef TARGET_MACHO

#define TARGET_MACHO 0
+#define MACHO_DYNAMIC_NO_PIC_P 0
#endif

Anyone have any objections to this?

Probably okay? Will let an ARM person say more. Just make sure that it
has the appropriate LLVM LOCAL markers.

-bw

That's what I originally had, but then backed it down to the version I
forwarded. Not sure why that worked for me and not you.

deep

Looks fine to me.

2009/3/17 Dale Johannesen <dalej@apple.com>

Can someone with llvm-gcc/ARM expertise (Dale?) please review Sandeep’s patch? The patch works for me in building an LLVM-based cross-compiler from x86_64/Linux to ARM/Linux.
I have been able to build without the patch to the asm file, but several people have reported needing the asm patch on http://llvm.org/PR2545 .

Can someone with llvm-gcc/ARM expertise (Dale?) please review Sandeep’s patch? The patch works for me in building an LLVM-based cross-compiler from x86_64/Linux to ARM/Linux.
I have been able to build without the patch to the asm file, but several people have reported needing the asm patch on http://llvm.org/PR2545 .

Looks OK to me. The asm file is the only thing I’d have reservations about (that isn’t marked as a local change, btw, and I don’t think it was), but the parentheses should work fine.

2009/3/18 Dale Johannesen <dalej@apple.com>

Can someone with llvm-gcc/ARM expertise (Dale?) please review Sandeep’s patch? The patch works for me in building an LLVM-based cross-compiler from x86_64/Linux to ARM/Linux.
I have been able to build without the patch to the asm file, but several people have reported needing the asm patch on http://llvm.org/PR2545 .

Looks OK to me.

Thanks for the review. Sandeep, I’ve committed your patch (with some modifications) to arm.h – please sync and see if this version works for you.

The asm file is the only thing I’d have reservations about (that isn’t marked as a local change, btw, and I don’t think it was), but the parentheses should work fine.

I haven’t yet submitted the .asm patch – I’m also curious how the /* LLVM LOCAL */ markers work in the Apple merges when it’s not an addition but rather a modification, e.g.:

  • THUMB_DIV_MOD_BODY 1
  • /* LLVM LOCAL */
  • THUMB_DIV_MOD_BODY(1)

Will Apple merger think that this is a new line added in the LLVM and complain that it’s missing a line without the parens from the Apple side, and end up inserting both? I admit I’m still a bit unclear as to how the auto-patcher works in all cases.

Would it make sense to do these fixes on the Apple side of the tree, so that Apple and LLVM are in agreement on this? It looks like a trivial change. Or I can make a patch with /* APPLE LOCAL */ markers so that Apple’s and public LLVM SVN don’t differ so much for such a change.

Let me know what you prefer.

Misha