Build Attributes Proposal

Hi all,

So, after long rounds to define how the build attributes are going to
be represented in IR, and after Jason has implemented build attributes
in MC, I have a proposal on how to represent this in IR.

First, we need to distinguish between target dependent and independent
attributes. Generic things like optimization level, specific
optimizations, use of vector instructions, use of exceptions,
floating-point compatibility can all go in to the generic list of
attributes. Local attributes like use ARM or Thumb, interoperability,
extensions, CPU name and architecture go into a target-specific
section.

Following the current rule in IR for specific attributes that are not
directly represented, and Chris' suggestion, I propose to use global
variables to represent them.

Representation:

The type has an identifier (Enum somewhere in LLVM) and a description or value:

%llvm.attribute = type { i32, i8* }

The generic attribute list can have, for instance, the optimization level used:

// suppose 1 is the key to "optimization level"
@llvm.attributes = appending global [2 x %llvm.attribute]
[%llvm.attribute { i32 1, i8* getelementptr inbounds ([3 x i8]* @.str,
i32 0, i32 0) }, %llvm.attribute zeroinitializer], align 4
@.str = private constant [3 x i8] c"O3\00"

And the target-specific has something like the CPU name and whether
it's using R9 as SB:

// 5 is CPU_name in ARM ABI, 14 is ABI_PCS_R9_use
@llvm.arm.attributes = appending global [2 x %llvm.attribute]
[%llvm.attribute { i32 5, i8* getelementptr inbounds ([10 x i8]*
@.str1, i32 0, i32 0) }, %llvm.attribute { i32 14, i8* null }], align
4
@.str1 = private constant [10 x i8] c"ARM946E-S\00"

Documentation:

It'd be good to have a list of all options, generic or otherwise. The
attributes enum could have a few lines explaining the options, but the
accepted values is more difficult to find in the code, so a
documentation (not LangRef) would be good, especially for the generic
part. The ARM options are already explained in the ABI documents and I
believe for other platforms as well.

Front-end/Back-End:

The front-end should be responsible for validating the
build-attributes, both for the generic and the specific parts, to make
sure at least that the identifiers are valid and their values in the
correct range (or in a given set of acceptable strings). The front-end
should not generate attributes that were not required, directly or
indirectly, but the user. Default behaviours should be already covered
by the back-end already, so no need to restate the obvious.

The back-end, and especially the linker, is free to assume whatever
they want to make the objects link without breaking any user requests.
The final executable should have only the build attributes necessary
to compile and link correctly.

I don't know how the back-end will get this, but there should be no
difference from the current implementation of other @llvm. globals.
There is already a place holder for ARM build attributes which will
have to be hooked to this values.

Use:

So, the biggest problem we have today in the LLVM back-end is that it
assumes too much.

If given ARMv7A, it assumes A8+NEON, which is not always true. Even if
given Cortex-A8 or A9 specifically, it still assumes NEON, which
again, is not 1 to 1. For instance, the new Tegra2 boards sport an A9
without NEON. LLVM generated code would probably break on them. While
putting a special case for Tegra2 would work, in ARM world, there are
more exceptions than rules, so that's not the optimal solution.

The ARM back-end doesn't print ".fpu" in the ASM file. GAS assumes
that you want NEON support if it sees NEON instructions in the code,
but not all linkers assume that much. The NEON instruction could be a
bug in the code generation pass, and you'd never notice if the linker
wouldn't complain about it.

These are the things that come to my mind now (as I've dealt with them
recently) but the number of examples are to great to list them here. I
hope to have conveyed the idea. :wink:

In the case of lack of build attributes, the back-end is free to
assume whatever it wants, but it should *always* honour the attributes
to give the user most flexibility.

Please let me know your thoughts so I can start looking into plugging
the ARM part in the ARM back-end.

Hi Renato,

So, after long rounds to define how the build attributes are going to
be represented in IR, and after Jason has implemented build attributes
in MC, I have a proposal on how to represent this in IR.

First, we need to distinguish between target dependent and independent
attributes. Generic things like optimization level, specific
optimizations, use of vector instructions, use of exceptions,
floating-point compatibility can all go in to the generic list of
attributes.

I don't see why you would want to put any of these things in the module
at all:

* optimization level: if you want the linker to optimize at -O3, I think
you should just pass -O3 to the linker. I don't see why the fact that
the original files were optimized at -O3, -O0, -O0 and -O3 (for example)
is relevant to link time optimization.

* specific optimizations: likewise.

* use of vector instructions: if you want the linker to codegen for SSE3,
then pass -msse3 to the linker.

* use of exceptions: if a file is compiled with -fno-exceptions then
all functions have the nounwind attribute and contain no invokes.
This is all that is needed to automatically have the code generators
not generate unwind info for them, so no special module level attribute
is needed, everything is there already.

* floating point compatibility: not sure what you mean by this.

Ciao,

Duncan.

Hi Duncan,

I'm not sure you're arguing against having global build attributes
(instead of just specific ones), or against the whole idea of having
build attributes.

I'll assume the former, as I take it that having support for ARM build
attributes in LLVM is important (for compatibility, at least).

* optimization level: if you want the linker to optimize at -O3, I think
you should just pass -O3 to the linker. I don't see why the fact that
the original files were optimized at -O3, -O0, -O0 and -O3 (for example)
is relevant to link time optimization.

* specific optimizations: likewise.

* floating point compatibility: not sure what you mean by this.

I don't have any specific optimization in mind, but some ARM build
attributes refer to the way optimizations and floating-point were
assumed when compiling.

If the user required IEEE 754 precision when compiling, you should
honour that, possibly using libraries that guarantee you won't diverge
from the standard.

Please read the Addenda for more information:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0045c/index.html
(chapter 2.3.7)

There could be similar assumption in Intel or PowerPC, they're not
exclusively on ARM, so having them on a global attribute list (that
can easily be ignored by any target), could help.

* use of vector instructions: if you want the linker to codegen for SSE3,
then pass -msse3 to the linker.

It's not always that simple. In some benchmarks or build systems you
can't easily fiddle with the linker parameters and the compilation is
done in multiple, separated steps, so it's also not easy to propagate
the user options as they don't share the same driver.

Also, a vectorized attribute would tell if you have to link with
vectorized libraries or normal ones, and if you have multiple styles
of vector code (mmx, sse, etc) you could choose which one, even if the
user haven't asked for a specific one.

AFAIK, GCC assumes that, if you have vectorized code, you want
vectorized libraries and vice-versa. But that's very simplistic. It
may occur that your code was so small that couldn't be vectorized, but
you still want vectorized libraries or vice-versa. While you can do
that by passing arguments to the linker, you may be in a situation
where you cannot pass arguments to the linker in the first place, or
you can't re-compile part of your program (no source?) and want to
enforce a different set of rules to the rest of the system.

* use of exceptions: if a file is compiled with -fno-exceptions then
all functions have the nounwind attribute and contain no invokes.
This is all that is needed to automatically have the code generators
not generate unwind info for them, so no special module level attribute
is needed, everything is there already.

Well, not generating unwind information for your functions doesn't
mean you can't have them in the libraries. If your code is not
expecting exceptions to be thrown, and you have libraries that don't
throw them, you can use the appropriate libraries for your code.

Also, if you have nounwind in one source but not in the other, the
compiler can't predict all the interoperability problems that will
come from that, if you compile all of them together. With such a build
attribute, the compiler might print a warning (or even an error) for
cases where there could be potential problems when intermixing
different assumptions.

See, I'm not enforcing a standard behaviour on the global
@llvm.attributes. I don't even need it, to be honest, but I didn't
want to propose such a system without accounting for the general case.

From my point of view, it's important to have ARM build attributes in

the front-end all the way to the back-end. If other platforms can use
the same encoding, better, if not, I don't mind.

Just having @llvm.arm.attributes would be fine for me...

cheers,
--renato

In fact they are more suitable as function attributes. For example, we already have function attributes for -Os.

I think missing a point here. Are you proposing that linker selects libraries based on attributes encoded in IR ?

Hi Devang,

I don't see why you would want to put any of these things in the module
at all:

In fact they are more suitable as function attributes. For example, we already have function attributes for -Os.

Well, if it's a compilation parameter (rather than a function
attribute), applying to all functions in a module doesn't make all
that sense, but I guess one could do that.

I was trying to avoid pollution of the LangRef and moving it to a
target specific namespace. You don't have to implement it in your
front-end if you don't want. In that case, the back-end will assume
all the defaults are to be selected and act as it would today. But if
the compiler want's to pass those options any time, they should be
allowed to and that should be carried throughout the compilation life
time (front to back end) and honoured by every pass.

I think missing a point here. Are you proposing that linker selects libraries based on attributes encoded in IR ?

That was just a top-of-my-head example but, yes. The behaviour of the
compiler and linker could change in face of the build attributes.

But, as I said, you're free to ignore it completely if you want. You
could have an ARM compatibility mode that only regards build
attributes when in this mode. That would make it much easier to
compile using ARM+LLVM tools.

We originally thought we could use metadata, but it would have to be
linked to a global variable, and there's no point of having a global
variable just for that.

cheers,
--renato

I think some of this discussion is missing the context. ARM has defined per-object file build attributes to record things like the optimization level, whether the code is ARM vs. Thumb, etc. This is not something new nor is it specific to LLVM. As far as I know, the linker is not obligated to do anything with this information, but compilers that follow ARM's specifications are expected to provide it.

Putting the build attributes on individual functions is not a good match for the final result, which is a single set of attributes for the entire object file. If you put separate attributes on the functions, then code gen will have to scan all the functions to see if they have the same attribute values.

It would help you solve the case where to module with different attributes are merged together.

However, Bob is correct, I was missing the context. Sorry about the confusion.

I think some of this discussion is missing the context. ARM has defined per-object file build attributes to record things like the optimization level, whether the code is ARM vs. Thumb, etc. This is not something new nor is it specific to LLVM. As far as I know, the linker is not obligated to do anything with this information, but compilers that follow ARM's specifications are expected to provide it.

Sorry about that, I didn't want to sound redundant after my previous
round of emails about build attributes... :wink:

Putting the build attributes on individual functions is not a good match for the final result, which is a single set of attributes for the entire object file. If you put separate attributes on the functions, then code gen will have to scan all the functions to see if they have the same attribute values.

Precisely.

cheers,
--renato

Hi Bob,

So, after long rounds to define how the build attributes are going to
be represented in IR, and after Jason has implemented build attributes
in MC, I have a proposal on how to represent this in IR.

First, we need to distinguish between target dependent and independent
attributes. Generic things like optimization level, specific
optimizations, use of vector instructions, use of exceptions,
floating-point compatibility can all go in to the generic list of
attributes.

I don't see why you would want to put any of these things in the module
at all:

In fact they are more suitable as function attributes. For example, we already have function attributes for -Os.

I think some of this discussion is missing the context. ARM has defined per-object file build attributes to record things like the optimization level, whether the code is ARM vs. Thumb, etc. This is not something new nor is it specific to LLVM. As far as I know, the linker is not obligated to do anything with this information, but compilers that follow ARM's specifications are expected to provide it.

Putting the build attributes on individual functions is not a good match for the final result, which is a single set of attributes for the entire object file. If you put separate attributes on the functions, then code gen will have to scan all the functions to see if they have the same attribute values.

thanks for the explanation. Perhaps you can use global asm statements for this
(see http://llvm.org/docs/LangRef.html#moduleasm )? These can be used to inject
arbitrary strings into the assembler. Dragonegg uses this to provide an "ident"
string like this:

   $ gcc-4.5 -fplugin=dragonegg.so -S -o - -flto empty.c
   ; ModuleID = 'empty.c'
   target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
   target triple = "x86_64-unknown-linux-gnu"

   module asm "\09.ident\09\22GCC: (GNU) 4.5.2 20101028 (prerelease) LLVM: 118890\22"

In the assembler output this gives:
   $ gcc-4.5 -fplugin=dragonegg.so -S -o - empty.c
     .file "empty.c"

     .ident "GCC: (GNU) 4.5.2 20101028 (prerelease) LLVM: 118890"

     .section .note.GNU-stack,"",@progbits

Ciao,

Duncan.

Hi Duncan,

This could work, but it's very limited. Besides, there's already an MC
build attribute infrastructure, one would have to interpret those
strings in the back-end to an MC construct then back again to ASM and
ELF.

I don't see why having a magic global would be worse than using module asm...

--renato

Hi Renato,

   module asm "\09.ident\09\22GCC: (GNU) 4.5.2 20101028 (prerelease) LLVM:
118890\22"

Hi Duncan,

This could work, but it's very limited. Besides, there's already an MC
build attribute infrastructure, one would have to interpret those
strings in the back-end to an MC construct then back again to ASM and
ELF.

I agree that it's limited. As for MC, it will need to handle these strings
anyway since this is an existing LLVM feature (coming from gcc originally)
that needs to be supported.

I don't see why having a magic global would be worse than using module asm...

Because magic globals need magic treatment, i.e. special logic in the code
generators etc. Why add all this new stuff if an existing feature will do?

Ciao,

Duncan.

I agree that it's limited. As for MC, it will need to handle these strings
anyway since this is an existing LLVM feature (coming from gcc originally)
that needs to be supported.

Do you mean that LLVM-GCC generates build attributes as asm strings in IR?

Because magic globals need magic treatment, i.e. special logic in the code
generators etc. Why add all this new stuff if an existing feature will do?

Build attributes are not asm specific, and implementing them in an asm
specific keyword seems kludge to me. Not to mention that you lock the
IR in a ARM-GNU-ASM specific mode, which is a dead-end. Also, passing
".build_attribute" strings in IR if you're producing ELF doesn't make
sense, IMO.

In a global array, the indexes are represented directly, only leaving
the associated string if additional information is necessary (like CPU
name, which is already a string).

cheers,
--renato

Hi Renato,

I agree that it's limited. As for MC, it will need to handle these strings
anyway since this is an existing LLVM feature (coming from gcc originally)
that needs to be supported.

Do you mean that LLVM-GCC generates build attributes as asm strings in IR?

no, I mean that gcc supports file level ASM, which is why llvm-gcc and LLVM
support it too, and which presumably means that MC will have to support it
one day.

Because magic globals need magic treatment, i.e. special logic in the code
generators etc. Why add all this new stuff if an existing feature will do?

Build attributes are not asm specific, and implementing them in an asm
specific keyword seems kludge to me. Not to mention that you lock the
IR in a ARM-GNU-ASM specific mode, which is a dead-end. Also, passing
".build_attribute" strings in IR if you're producing ELF doesn't make
sense, IMO.

It is hard for me to comment because I don't know anything about these
attributes. However, presumably they need to end up in the .s file.
I'm pointing out that you can put anything you like into the assembler
via module level asm. I'm not sure what you mean by "build attributes
are not asm specific". I never suggested they were! The so-called asm
is just a directive saying: place this string as-is in the final assembler.
I'm also not sure what ELF has to do with anything.

In a global array, the indexes are represented directly, only leaving
the associated string if additional information is necessary (like CPU
name, which is already a string).

Since I don't know anything about these attributes, talk of indexes and
so forth goes straight over my head.

All I'm saying is that if you want a series of strings to end up in the
.s file, this can be done using an existing LLVM feature.

Ciao,

Duncan.

Hi Duncan,

I assumed you knew what build attributes were in the first place, my mistake.

For a full explanation of what they are, see ARM ABI Addenda, section
2.1 (PDF). (Google's first hit for "arm abi addenda").

Simply put, build attributes express the user explicit intentions when
compiling code, so the linker can take better decisions when linking
the objects. These attributes apply normally to whole units, but could
apply to sections or even symbols, to express the compatibility or
preference towards specific components. With build attributes, it's
easier to pass information between tool chains, as some choices might
be obvious within the tool chain, but different across.

For example, the LLVM tool chain assumes that every Cortex-A8 has
NEON, while the ARM tool chain does not. So, if you compile a file
with Clang for Cortex-A8 and link with LLVM linker (or LD), everything
works fine, because that assumption is in every step of the tool
chain. However, if you try to link with armlink, it won't work, as the
NEON attribute (Tag_Advanced_SIMD_arch) will not be there, and when it
finds vector instructions, it'll bail out.

Another example I quote from the ABI:

"The compiler might choose to use only the Thumb ISA in a specific
build of a source file.
Nonetheless, the relocatable file should be tagged as having been
permitted to use the ARM ISA so that a linker can later link it with
ARM-state library code and generate ARM-state intra-call veneers if
that gives benefit to the executable file.
On the other hand, if the user intends code to be executed by both
ARM7TDMI and Cortex-M3, the compiler must be constrained to generate
only Thumb v1 instructions and the relocatable file should be tagged
as not permitted to use the ARM ISA."

So, as you can see, this gives the user and library writers freedom to
optimize and specialize their code to perfection. In a highly
segmented market such as ARM's, where every manufacturer has its own
ways to connect the components or build SoCs, this is the only way to
completely customize the tool chain to a completely customized
architecture.

Also, it's not something restricted to assembly only, but it's also
stored in ELF files (global, sections, symbols). This are also well
documented, long-lasting definitions that all ARM ABI compatible
compilers should honour when producing and consuming object files.
Specific manufacturer can also come up with a new set of attributes
that are specific to their own architecture, and as long as they don't
collide with the public definitions, other manufacturers are free to
ignore.

cheers,
--renato

Hi Renato, everyone

Since I don't know anything about these attributes, talk of indexes and
so forth goes straight over my head.

Hi Duncan,

I assumed you knew what build attributes were in the first place, my mistake.

For a full explanation of what they are, see ARM ABI Addenda, section
2.1 (PDF). (Google's first hit for "arm abi addenda").

Simply put, build attributes express the user explicit intentions when
compiling code, so the linker can take better decisions when linking
the objects. These attributes apply normally to whole units, but could
apply to sections or even symbols, to express the compatibility or
preference towards specific components. With build attributes, it's
easier to pass information between tool chains, as some choices might
be obvious within the tool chain, but different across.

For example, the LLVM tool chain assumes that every Cortex-A8 has
NEON, while the ARM tool chain does not. So, if you compile a file
with Clang for Cortex-A8 and link with LLVM linker (or LD), everything
works fine, because that assumption is in every step of the tool
chain. However, if you try to link with armlink, it won't work, as the
NEON attribute (Tag_Advanced_SIMD_arch) will not be there, and when it
finds vector instructions, it'll bail out.

Another example I quote from the ABI:

"The compiler might choose to use only the Thumb ISA in a specific
build of a source file.
Nonetheless, the relocatable file should be tagged as having been
permitted to use the ARM ISA so that a linker can later link it with
ARM-state library code and generate ARM-state intra-call veneers if
that gives benefit to the executable file.
On the other hand, if the user intends code to be executed by both
ARM7TDMI and Cortex-M3, the compiler must be constrained to generate
only Thumb v1 instructions and the relocatable file should be tagged
as not permitted to use the ARM ISA."

So, as you can see, this gives the user and library writers freedom to
optimize and specialize their code to perfection. In a highly
segmented market such as ARM's, where every manufacturer has its own
ways to connect the components or build SoCs, this is the only way to
completely customize the tool chain to a completely customized
architecture.

Also, it's not something restricted to assembly only, but it's also
stored in ELF files (global, sections, symbols). This are also well
documented, long-lasting definitions that all ARM ABI compatible
compilers should honour when producing and consuming object files.
Specific manufacturer can also come up with a new set of attributes
that are specific to their own architecture, and as long as they don't
collide with the public definitions, other manufacturers are free to
ignore.

I think there are several common use cases for these attributes -
1. to mark what ISA a function is compiled for: (i.e. is it a
thumb16/thumb32/ARM?)
2. to constrain the generated code to use specific co-processors.

At least for case 2, the ARMTargetMachine/ARMSubtarget contains a
distinct set of flags which should be output in the file scope section
of the attributes
For case 1, I don't know how common it is for the same executable to
cross thumb boundaries - (given that16bit thumb has limits on register
usage etc..) but I suppose its possible.

For now, I agree with Renato that using the Module-Level Inline
Assembly is probably not the right way to go - for example, some of
the messyness of .cpu asm statement mapping to many FileScope
attributes is the primary cause of the (likely) not-clean looking code
that needs to go into the current .ARM.attributes emitter for ELF.

What I would like to find out is the current set of supported ASM
attributes by the current AEABI compiler. Once *that* info is
available, we can go about sensibly factoring the required behavior in
LLVM -

Just my 2c.

-jason

At least for case 2, the ARMTargetMachine/ARMSubtarget contains a
distinct set of flags which should be output in the file scope section
of the attributes

Yes, getting the default behaviour from SubTarget is the easy part
(mostly done, now, I believe).

For case 1, I don't know how common it is for the same executable to
cross thumb boundaries - (given that16bit thumb has limits on register
usage etc..) but I suppose its possible.

It is possible and often used.

What I would like to find out is the current set of supported ASM
attributes by the current AEABI compiler. Once *that* info is
available, we can go about sensibly factoring the required behavior in
LLVM -

I can't make out which "AEABI compiler" you're talking about, but yes,
I agree with your approach.

There are essentially two "types" of attributes:

1. Hardware compatibility: arm/thumb, softfp/vfp/neon, fp precision,
registers to use, etc. These are more likely grabbed from the
SubTarget and changed according to the command-line options. We can
get away for the moment just by using the target descriptions and
automatically filling build attributes (without changing the IR).
Specific targets, like NVidia's Tegra2 (Cortex-A8 without NEON) will
break, though.

2. Binary compatibility: procedure call standard to use (apcs, aapcs,
vfp), size of wchar_t and enums. It's not crystal clear what
determines all of these parameters, so the user aspect (cmd-line
options) is more important in this case, and they should be honoured.
This is a much more complicated issue, but the IR already has some
information (like function attributes) that help in that matter.

So, for the moment, without changing the IR would get us far enough.
But some of these attributes will have to be recorded, eventually, and
that was the motivation of this preliminary discussion on how to
represent them in IR.

The list of build attributes supported by GCC must be hidden in its
own source code somewhere. The options that will, eventually (I hope),
set them are:

As you see there isn't a 1-to-1 relationship between command-line
options, target description and build attributes, and linkers are free
to interpret them (at least the binary compatibility ones) as they see
fit.

cheers,
--renato

PS: The size of wchar_t and enums are also in line with the size and
alignment for unions and bitfields we were talking about earlier, thus
why I thought that there could be a global attribute (@llvm.attribute)
rather than just an ARM specific one (@llvm.arm.attribute).

Hi Duncan,

This could work, but it's very limited. Besides, there's already an MC
build attribute infrastructure, one would have to interpret those
strings in the back-end to an MC construct then back again to ASM and
ELF.

I don't see why having a magic global would be worse than using module asm...

So, you mentioned that there should be no attributes that were not
requested by the users, right? If that is so, can't you produce the
asm in clang or you FE and have llvm never introduce new ones? LLVM
itself doesn't use the attributes, so it doesn't need to interpret
them.

--renato

Cheers,
Rafael

Right, we do similar things for Objective-C metadata that there is no reason to represent in IR, the objc frontend (like clang) just produces module-level inline assembly blobs.

-Chris

Another use for build attributes would be as a means to record the
build flags selected for each translation unit so that LTO could know
how to optimize/tune the result. This use seems more important to
solve than the ARM attributes under discussion here.

deep