I’ve been playing around with clang/LLVM looking at adding partial support for the draft technical report for embedded C extensions (TR18037, http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1169.pdf), specifically named address spaces.
Named address spaces need to be tracked in LLVM in essentially all the same places that alignment is tracked, which necessitates adding the information to the .bc format. Given the Apple has shipped .bc files I’m guessing that backwards compatibility is very important. Given this and the work I see happening on using the newish serialize/deserialize infrastructure what is the pattern for extending the .bc format in a backwards compatible way? Is it safe to add records to the writer for an instruction and predicate parts of the reader based on the number of records present so that old .bc files with fewer records for that entry should still be able to be read?
It's easy enough to extend a bitcode record in a compatible manner.
• The writer should place new fields only at the end of a record. Earlier readers will ignore them.
• If a record comes up short, a backwards-compatible default should be selected by the reader.
This provides backwards and forwards compatibility, which is great, and surprisingly simple to accomplish.
Sounds like you're adding fields to load and store nodes. To reduce the cost for programs that do not use memory spaces, you might try to optimize the representation by taking advantage of the default value when encoding the record.
FYI, there are no current plans to replace the implementation of the LLVM bitcode reader/writer with something that uses the new serialize/deserialize infrastructure. It is possible, however, that it could be used as a convenient tool to add new kinds of records to the bitcode.
The serialize/deserialize infrastructure is intended to be another API that sits just above the Bitstream reader/writer (which the LLVM bitcode reader/writer is built on), and its role is to serialize arbitrary objects using logic provided via C++ trait classes. The serializer keeps track of pointers and references (allowing objects with multiple pointers to them to be safely and transparently serialized, or even cyclic data structures). The serializer also allows almost complete transparency of the underlying bitstream format (including the notion of blocks and records), although the goal is to provide an interface to those details should the client need it (this is gradually taking form).
The serializer's big role right now is to support serialization of data structures in the new C frontend. This includes ASTs, and all the supporting meta data needed to serialize out a C program and read it back in. We currently have made a good deal of progress on this project. Thus initially, our goals with the serializer don't have to contend with problems of backwards compatibility with an existing application. Our goal is to first get serialization "right" for clang, but at the same time it isn't being engineered as an API that will only be useful in the new frontend.
Others addressed the other questions, one (surprising?) thing I’d recommend:
Unlike alignment and volatility, I think that the address space qualifier should be represented explicitly in the type system. The reason for this is primarily that pointers to different address spaces are really very different sorts of beasties: for example they can be codegen’d to have different sizes. Any property that affects how the value is stored in registers needs to be in the type instead of on the load/store instruction. Also, unlike volatile, it is not common to cast a pointer between two different address spaces.
The good thing about this is that I think it will make it substantially easier to update the various llvm optimizations if you do this. The meat of project boils down to adding a new address space qualifier field to PointerType, making sure PointerType takes this field into account when it is being uniqued, and adding the address space qualifier to things like global variable.
I’ve been playing around with clang/LLVM looking at adding partial support for the draft technical report for embedded C extensions (TR18037, http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1169.pdf), specifically named address spaces.
Named address spaces need to be tracked in LLVM in essentially all the same places that alignment is tracked,
Others addressed the other questions, one (surprising?) thing I’d recommend:
Unlike alignment and volatility, I think that the address space qualifier should be represented explicitly in the type system. The reason for this is primarily that pointers to different address spaces are really very different sorts of beasties: for example they can be codegen’d to have different sizes.
Very true.
Any property that affects how the value is stored in registers needs to be in the type instead of on the load/store instruction. Also, unlike volatile, it is not common to cast a pointer between two different address spaces.
Though perhaps infrequent, casting between address spaces is allowed based on rules that the target defines indicating which address spaces are subsets of others. Does supporting those casts require an explicit operation (ie intrinsic)?
The good thing about this is that I think it will make it substantially easier to update the various llvm optimizations if you do this.
Bonus!
The meat of project boils down to adding a new address space qualifier field to PointerType, making sure PointerType takes this field into account when it is being uniqued, and adding the address space qualifier to things like global variable.
Does this sound reasonable?
That sounds like it should be easier than adding the address space ID to all the instructions and SDNodes.
I’ll give it a try and see what happens. I can see that adding it to the type system makes it easier on the optimizer, but I don’t yet understand all the consequences for the code generator.
Any property that affects how the value is stored in registers needs to be in the type instead of on the load/store instruction. Also, unlike volatile, it is not common to cast a pointer between two different address spaces.
Though perhaps infrequent, casting between address spaces is allowed based on rules that the target defines indicating which address spaces are subsets of others.
Right.
Does supporting those casts require an explicit operation (ie intrinsic)?
This should just be a bitcast from one pointer to another pointer type.
The meat of project boils down to adding a new address space qualifier field to PointerType, making sure PointerType takes this field into account when it is being uniqued, and adding the address space qualifier to things like global variable.
Does this sound reasonable?
That sounds like it should be easier than adding the address space ID to all the instructions and SDNodes.
I'll give it a try and see what happens. I can see that adding it to the type system makes it easier on the optimizer, but I don't yet understand all the consequences for the code generator.
I don't know what the right answer is for codegen either. I'd suggest getting the llvm ir self-consistent, then worrying about it. Until we have a concrete machine that needs it, it is hard to anticipate what will be needed. In an ideal world, we'll just need flags on load/store nodes because the pointer registers will already be lowered to some other regclass.
Any property that affects how the value is stored in registers
needs to be in the type instead of on the load/store instruction.
Also, unlike volatile, it is not common to cast a pointer between
two different address spaces.
Though perhaps infrequent, casting between address spaces is allowed
based on rules that the target defines indicating which address
spaces are subsets of others.
Right.
Does supporting those casts require an explicit operation (ie
intrinsic)?
This should just be a bitcast from one pointer to another pointer type.
Good.
The meat of project boils down to adding a new address space
qualifier field to PointerType, making sure PointerType takes this
field into account when it is being uniqued, and adding the address
space qualifier to things like global variable.
Does this sound reasonable?
That sounds like it should be easier than adding the address space
ID to all the instructions and SDNodes.
I’ll give it a try and see what happens. I can see that adding it to
the type system makes it easier on the optimizer, but I don’t yet
understand all the consequences for the code generator.
I don’t know what the right answer is for codegen either. I’d suggest
getting the llvm ir self-consistent, then worrying about it. Until we
have a concrete machine that needs it, it is hard to anticipate what
will be needed.
Indeed.
In an ideal world, we’ll just need flags on load/
store nodes because the pointer registers will already be lowered to
some other regclass.
I assume malloc’s and memcpy’s would need them as well?
In an ideal world, we'll just need flags on load/
store nodes because the pointer registers will already be lowered to
some other regclass.
I assume malloc's and memcpy's would need them as well?
Yeah, seems likely. I think malloc gets lowered in SDISel to a call though, so maybe it doesn't need it. I'm not sure what "malloc from alternate address space" really means though. There can only be one function named "malloc" :). Maybe LangRef should explicitly ban the malloc instruction from returning a pointer to an alternate address space?
In an ideal world, we’ll just need flags on load/
store nodes because the pointer registers will already be lowered to
some other regclass.
I assume malloc’s and memcpy’s would need them as well?
Yeah, seems likely. I think malloc gets lowered in SDISel to a call
though, so maybe it doesn’t need it. I’m not sure what “malloc from
alternate address space” really means though. There can only be one
function named “malloc” :).
That’s true. I guess I was thinking of “intrinsic” overloading, but you can’t really do that based on return type, can you! Well, you could, but it’d be odd.
Maybe LangRef should explicitly ban the
malloc instruction from returning a pointer to an alternate address
space?
Yes. I think that, in embedded C parlance, malloc should always return a pointer to the generic address space. If casting that pointer to another address space is valid given the target’s rules, that’s fine.
This should just be a bitcast from one pointer to another pointer type.
Here's a likely counterexample: the AVR (8 bit embedded) has a
byte-addressed data space and word-addressed code space.
Legal C code involving pointer casts can confuse all known versions of
avr-gcc into generating bad function pointer code that jumps to twice the
address of the intended target. Last I heard there's no real plan to fix
this.
While the embedded C standard permits a way out where the compiler
understands that code space and data space pointers are different, I don't
see any way that pointer casts (for example, from an unqualified pointer
to either code ptr or data ptr) can be nops.
This should just be a bitcast from one pointer to another pointer type.
Here's a likely counterexample: the AVR (8 bit embedded) has a
byte-addressed data space and word-addressed code space.
ok, presumably these would be two address spaces then.
While the embedded C standard permits a way out where the compiler
understands that code space and data space pointers are different, I don't
see any way that pointer casts (for example, from an unqualified pointer
to either code ptr or data ptr) can be nops.
I'm just talking about the representation of the code in the LLVM IR, how the bitcast from one memory address space to the other is code generated should be up to the target.
Any suggestions on type syntax in .ll files for address spaced pointers?
I was thinking postfix of the type name, but I’m up in the air about what a good separator would be. Simply whitespace?
I’ve come across a hitch. Store instructions do not reference the pointer type in the .bc format, only the stored type. The .bc reader constructs the pointer type from the stored value’s type. This means that the address space information doesn’t come along for the ride.
I see three solutions:
Change how stores are written/read in .bc to store the pointer type rather than the stored type. This is the most straight forward, but I think it also breaks .bc compatibility in a way that’s impossible to work around. There’s no way to differentiate the new and old forms.
Have an extended record form of stores that carries the address space information for the pointer type which then gets restored by the reader. This preserves backwards compatibility, but is kind of ugly.
Store address space information on all types (not just pointers), but it only really affects how pointers are handled. This ensures that address spaces go wherever the type goes. This is pretty invasive, and I’d like to avoid that overhead if at all possible.
My suggestion would be 2 for now with an intention to change to 1 in LLVM 3.0.
I'd prefer to make it really obvious what is going on in the .ll file. How about something like:
%x = load i32 * addrspace(42) %ptr
?
Putting the marker after the * it applies to should make the .ll file easier to parse, but putting it *immediately* before the '*' is also fine with me. This would give:
Unlike alignment and volatility, I think that the address space qualifier should be represented explicitly in the type system. The reason for this
I've come across a hitch. Store instructions do not reference the pointer type in the .bc format, only the stored type. The .bc reader constructs the pointer type from the stored value's type. This means that the address space information doesn't come along for the ride.
Ah, that is annoying.
I see three solutions:
1) Change how stores are written/read in .bc to store the pointer type rather than the stored type. This is the most straight forward, but I think it also breaks .bc compatibility in a way that's impossible to work around. There's no way to differentiate the new and old forms.
I strongly prefer this approach. Implementing this without breaking old .bc files is actually pretty simple. Just add a new "FUNC_CODE_INST_STORE2" record, and define it however you want (with a new, previously unused, ID #).
The reader should read both FUNC_CODE_INST_STORE (which can't involved addr spaces) and FUNC_CODE_INST_STORE2 (which can). The .bc writer can switch to unconditionally writing out stores in FUNC_CODE_INST_STORE2 format.
Please add a generous block comment to llvm/include/llvm/Bitcode/LLVMBitCodes.h above the new enum explaining what the difference is though.
Unlike alignment and volatility, I think that the address space qualifier
should be represented explicitly in the type system. The reason for this
I’ve come across a hitch. Store instructions do not reference the pointer
type in the .bc format, only the stored type. The .bc reader constructs the
pointer type from the stored value’s type. This means that the address space
information doesn’t come along for the ride.
Ah, that is annoying.
I see three solutions:
Change how stores are written/read in .bc to store the pointer type rather
than the stored type. This is the most straight forward, but I think it also
breaks .bc compatibility in a way that’s impossible to work around. There’s
no way to differentiate the new and old forms.
I strongly prefer this approach. Implementing this without breaking old
.bc files is actually pretty simple. Just add a new
“FUNC_CODE_INST_STORE2” record, and define it however you want (with a
new, previously unused, ID #).
The reader should read both FUNC_CODE_INST_STORE (which can’t involved
addr spaces) and FUNC_CODE_INST_STORE2 (which can). The .bc writer can
switch to unconditionally writing out stores in FUNC_CODE_INST_STORE2
format.
Please add a generous block comment to
llvm/include/llvm/Bitcode/LLVMBitCodes.h above the new enum explaining
what the difference is though.
case bitc::TYPE_CODE_POINTER: // POINTER: [pointee type]
if (Record.size() < 1)
return Error("Invalid POINTER type record");
ResultTy = PointerType::get(getTypeByID(Record[0], true));
break;
The interesting point here is that PointerType currently has one field (that it push_back's onto TypeVals). The reader requires this field to be present, but ignores any additional fields. You should just be able to add another typeVals.push_back() to the writer, and the readers will be transparently compatible with it. Just make the new reader do something like this:
case bitc::TYPE_CODE_POINTER: // POINTER: [pointee type]
if (Record.size() < 1)
return Error("Invalid POINTER type record");
unsigned AddrSpace = Record.size() >= 2 ? Record[1] : 0;
...
Christopher,
It has been a while since we last talked about C embedded extensions in
LLVM, I was moving back and froth from project to project and didn't get
a chance to follow up. I was wondering if you have made any changes to
LLVM IR and if so what has been added. And how can I contribute?
My understanding is that Christopher's patches have all landed in llvm, so the IR is capable of capturing and propagating the address space information. However, we have no front-end that correctly generates this. My understanding is that Christopher has patches in progress to add this to clang, but I'm not sure what the state of these is. It would be great to get have help getting this into clang.
