Let me try to explain the issue around typed memory. It’s orthogonal to TBAA. Plus I agree the allocation type is irrelevant; it’s the type of the load/store operations that matters.
Fact 1: LLVM needs an “universal holder” type in its IR.
- Frontends load data from memory and don’t necessarily know the type they are loading. When you load a char in C, it can be part of a pointer or an integer, for example. Clang has no way to know.
- That’s necessary to express implementations of memcpy in IR, as it copies raw data without knowing what the type is.
- LLVM lowers small memcpys into load/store.
I hope you agree LLVM needs a type that can contain any other type.
Fact 2: optimizations for this “universal holder” type need to be correct for any LLVM IR type.
Since this type can hold anything, optimizations that apply to this type must be correct for integers, pointers, etc. Otherwise, the results would be incorrect if the data was cast to the “wrong” type.
The corollary of this fact is that GVN is not correct for the “universal holder” type in general. That’s because GVN is not correct for pointers in general (though correct in some cases). GVN doesn’t work for pointers since just because 2 pointers compare equal, it doesn’t mean they have the same provenance. As LLVM’s AA uses provenance information, this information must be preserved.
Hopefully you agree with everything I wrote so far. If not please stop me and ask.
Ok, so what’s the solution?
We have two candidates for this “universal holder” type:
- integer (i**)
- byte (b**), a new dedicated type
The advantage of going with integers is that they already exist. The disadvantage? To make LLVM correct, we would need to disable GVN in most cases or stop using provenance information in AA (make pointers == integers).
This sounds pretty bad to me due to the expect performance hit.
Introducing a new type requires work, yes. But it allows us to keep all integer optimizations as aggressive as we want, and only throttle down when encountering the byte type. Another benefit is that loading a byte from memory doesn’t escape any pointer, while with the first solution you need to consider all stored pointers as escaped when loading an integer.
Introducing the byte type allow us to fix all the integer/pointer casts bugs in LLVM IR. And it enables more optimizations as we won’t need to be careful because of the LLVM IR bug.
A 3rd option is to keep everything as-is. That is, keep wrong optimizations in LLVM and keep adding hacks to limit the miscompilations in real-world programs.
That has bitten us multiple times. Last time someone told me my miscompilation example was academic, LLVM ended up miscompiling itself, and then miscompiled Google’s code. Only this year we are planning to fully fix that bug (we have a hack right now).
Last week Alive2 caught a miscompilation in the Linux kernel, in the network stack. The optimization got the pointer arithmetic wrong. Pretty scary, and the bug may have security implications.
Anyway, just to argue that leaving things as-is is not an option. We have momentum and 3 GSoC students ready to help to fix the last few design bugs in LLVM IR.
I know this stuff is complex; please ask questions if something is not clear or if you disagree with anything I wrote above.