Dear community,
we are currently facing a problem related to the new type system in llvm 3.0.
Our setting is the following: We have two or more modules, all in the same LLVMContext. They are sharing some types, meaning that for example functions in different modules are referencing the same (meaning pointer identical) type.
Now we write the different modules to the disk, and read them back from another program (again into the same LLVMContext).
The problem now is that named structs get duplicated for the different modules, meaning that when the second module is read, a new named struct is created in the context, and its name gets suffixed by a number.
This is because each module contains its own type table with all the types used in that module. When reading in the corresponding bitcode, the BitcodeReader explicitly calls StructType::create, without looking up in the context whether an equivalent type (even with the same name) already exists.
So I think that llvm is behaving correctly here, according to the new type system. But for us, the problem is that previously identical types are not identical any more after deserialization, which leads to problems when copying code between the modules.
So did anyone already stumble across that problem, and solved it? Or is there a known solution to it?
Our idea for solving this is to add a named metadata node to each module before serializing it to bitcode, in order to identify previously identical types after deserialization. The metadata consists of a list of constants, where each even entry is a ConstantAggregateZero of a named struct, and the succeeding entry is a constant integer uniquely identifying that type. We plan to just use the Type* casted to i64.
Then after reading in all modules, we could find the named metadata, iterate over its elements and unify all Types which have the same number assigned. This would involve recreating and replacing global values, if their type changed.
Does this approach sound reasonable to you?
Another option would be to merge all modules together in a new module before serialization, prefixing all global values. After deserialization of this single module, the types would still be correct, and the module could be split up again. But this would require some rearrangements in our code since the modules would have to be written out at one single point. That's why we discarded that idea for now.
If anything is unclear, I can provide examples.
Thanks for any comments,
Clemens
This is because each module contains its own type table with all the types used in that module. When reading in the corresponding bitcode, the BitcodeReader explicitly calls StructType::create, without looking up in the context whether an equivalent type (even with the same name) already exists.
So I think that llvm is behaving correctly here, according to the new type system. But for us, the problem is that previously identical types are not identical any more after deserialization, which leads to problems when copying code between the modules.
So did anyone already stumble across that problem, and solved it? Or is there a known solution to it?
I'm familiar with the scenario, but haven't heard of anyone trying to do something quite like this. The linker has to solve the exact same problem (read multiple .bc files and unify types across them). This is the impetus behind TypeMapTy in lib/Linker/LinkModules.cpp. You'll probably need to do something like that.
Our idea for solving this is to add a named metadata node to each module before serializing it to bitcode, in order to identify previously identical types after deserialization. The metadata consists of a list of constants, where each even entry is a ConstantAggregateZero of a named struct, and the succeeding entry is a constant integer uniquely identifying that type. We plan to just use the Type* casted to i64.
Then after reading in all modules, we could find the named metadata, iterate over its elements and unify all Types which have the same number assigned. This would involve recreating and replacing global values, if their type changed.
Does this approach sound reasonable to you?
I have to ask: why are you writing these modules out as separate bc files? A more typical approach would be to write out one big .bc file, and then lazily read in functions as you need them. This avoids problems like you're seeing, and has the advantage of sharing types and constants as well.
-Chris
Hi Chris,
thanks for your answer!
The linker has to solve the exact same problem (read multiple .bc files and unify types across them). This is the impetus behind TypeMapTy in lib/Linker/LinkModules.cpp. You'll probably need to do something like that.
I already looked into that. The linker is using the GlobalValues of both modules to identify the types to unify.
This leads to interesting effects in some cases, but I'll write another post about this.
Nevertheless the TypeMapTy is a great piece of code, and we will definitely reuse it to remap duplicated types (and composed types) to unique ones (via mutateType(), recursively descending to all uses).
I have to ask: why are you writing these modules out as separate bc files?
I knew that someone would ask that 
We need to have separate modules during runtime. One of them contains the code that is actually JIT compiled and executed, and simultaneously different optimizations are concurrently (in individual threads) building up or restructuring new code in their individual "working modules". Eventually some code will get copied over to the main module to be executed, and that's why they need to use the same types.
Cheers,
Clemens
I think that the linker would indeed be able to remap the types correctly, but the main obstacle here is that we only copy individual functions. Maybe it would be possible to first copy the function into a new module, and then link this into the main module, but then again the question is how to correctly copy the function to the temporary module
Another thing is that for the transformations, it would be much nicer if the types in all working modules are the same as in the main module.
This copying of individual functions between modules works very well *if* the used types are the same. All that had to be changed is the remapping, because we have to remap the global values as well. We are basically using a slightly extended version of the MapValue function.
So let's concentrate on how to get the types unique
Cheers,
Clemens
Ah, thanks, I didn't know that tool.
But I don't think it's applicable here. It's a command-line tool which removes all global values and functions which were not explicitely stated on the command line, and then dumps the remaining module.
Are you proposing to
- compute the set of referenced global values and functions from the individual function
- write the working module to the disk
- run llvm-extract
- read the extracted module
- use the linker to link it into the main module?
Then I think it's easier to use our current implementation, and just fix the issue with the duplicated types.
Sorry if I misunderstood your suggestion.
Clemens
Hi Clemens,
...
what do you mean by "copied over to the main module"? If you want to add
additional IR to the main module then you should link it in using the
linker.
I think that the linker would indeed be able to remap the types
correctly, but
the main obstacle here is that we only copy individual functions.
Maybe it would
be possible to first copy the function into a new module, and then
link this
into the main module, but then again the question is how to correctly
copy the
function to the temporary module
llvm-extract?
Ah, thanks, I didn't know that tool.
But I don't think it's applicable here. It's a command-line tool
all the LLVM command line tools are wrappers for functionality in the LLVM
libraries. If you look at the code for llvm-extract you will see that it is
about 150 lines long or so, and pretty straightforward. I'm suggesting you do
something like what that tool does to extract your function into a new module
in some appropriate way given your particular setup (that code would probably
be pretty close to what you are doing right now), then use the linker library
to link the new module into the target one. I'm not saying that this is the
only possibility, but it leaped to mind when I read your email, which seemed
to me to say "I'm trying to link a function into a module without using the
linker but it doesn't work well because we don't do XYZ that the linker does".
Ciao, Duncan.
which removes