[RFC] Adding opaque types to LLVM IR

,

Background

In a previous thread, I discussed the possibility of adding an opaque type to LLVM for the purposes of representing certain SPIR-V types better in a world of opaque pointers. Given that there was interest in such a type, I would like to discuss a more specific, fleshed-out proposal for such types.

In SPIR-V, we have a family of types that represent various opaque hardware types (such as an OpTypeImage) that need to be preserved through all optimizations and be emitted as specific types in the output SPIR-V file. These types are represented as pointers-to-opaque-structs in the current versions of LLVM IR, but optimizations will sometimes introduce illegal optimizations on these types, such as ptrtoint/inttoptr.

Furthermore, it is not always possible to correctly identify that a given ptr value in LLVM IR refers to one of these types. Presently, types are inferred in opaque pointer mode by demangling function names, but return types in particular are usually not encoded in Itanium name mangling. Given that these types have very restricted usability in SPIR-V IR (it is not even possible to bitcast to/from these types), being unable to identify these types proves catastrophic for compilation.

Proposed Semantics

Opaque types are a new kind of fundamental LLVM type. The C++ API for this type, at least as far as type creation looks, would be as follows:

class OpaqueType : public Type {
public:
  static void get(LLVMContext &Ctx, StringRef Name);
  static void get(LLVMContext &Ctx, StringRef Name,
                  ArrayRef<Type *> Types, ArrayRef<unsigned> Parameters);
};

In LLVM IR terms, this would be written as:

opaque(“spirv.Image”, void, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0)
opaque(“spirv.Queue”)

Two opaque types are considered the same type if their Name, Types, and Parameters all compare equal.

Opaque types would be first-class types. It is legal to use them as function parameters or return values, in selects and PHIs. It would also be a sized type—for data layout purposes, the size would be the same as ptr addrspace(0)—which means it could be embedded in structs, used in allocas, loads, and stores, even used as global values. The constant values zeroinitializer, poison, and undef are all legal values for opaque types. What cannot be done is to convert between different opaque types—it is not legal to bitcast to or from an opaque type, and the following code would have target-defined behavior (which could be undefined behavior):

ptr @evil(opaque(“atype”) %val) {
  %memory = alloca opaque(“atype”)
  store opaque(“atype”) %val, ptr %memory
  %res = load ptr, ptr %memory
  return ptr %res
}

Values of opaque types cannot be generally introspected by target-independent passes. These values are expected to mostly arise from target-specific intrinsics, and optimizations on those intrinsics are possible based on existing LLVM attributes (e.g., readonly, speculatable). Furthermore, it is possible (indeed, desirable) for optimization passes like SROA to convert allocas of opaque type to SSA values, or for DCE to eliminate unused, side-effect-free instructions of opaque type.

The meanings of opaque types are determined by the target. Which semantics the opaque type has is determined by the target, and I would hope that targets document the expected opaque types and their semantics (LLVM target documentation tends to be woefully lacking in this regard, and I would like to set higher standards here). For SPIR-V—which is the main motivating case for me—the expected opaque types would have the string name be “spirv.*TYPE*” (where TYPE is one of the OpType* in SPIR-V that doesn’t readily correspond to any other LLVM IR type), with extra parameters being necessary iff the SPIR-V OpType* requires them.

Impact on optimizations

While I haven’t implemented the proposal in its entirety, I have implemented enough of it to be confident that the proposal would service my needs. I predict that the impact of these changes on existing optimizations is likely to be relatively small—the only optimizations I needed to fix were SROA and GVN, which both have independent methods that amount to “can type T1 be bitcast to T2” that tends to default to “yes”. It’s possible there are more such calls in the codebase that haven’t triggered on my test suites yet, but I haven’t found any other issues in existing target-independent optimizations.

Unanswered Questions

Naming of opaque types

In the previous discussion, the general design has been to give a name to these types that’s akin to the struct name, such as:

%imgf2d = opaque(“image”, float, i32 2, i32 42)

There are some advantages to having syntax like this (see below). In my test implementation, I’ve avoided this because I never tested opaque types beyond a single string parameter, and having two names for the same thing seemed a bit much. Given that I expect the extra parameters on opaque types are likely to be relatively rare, I’m not sure it’s worth the extra confusion of having an extra name and having to work out the degree to which that name matters.

Moving existing types to opaque types

When preparing this RFC, I noticed that x86_mmx and x86_amx already have semantics very similar to opaque types (more so for x86_mmx than x86_amx). It may be possible to move these types to being x86-specific opaque types, but doing so would necessitate adding target-specific hooks to work correctly, as the defaults for opaque types suggested in this RFC would be woefully incorrect. Even if these hooks were present, though, it may be too much code churn to move existing LLVM types to an opaque type model, and I don’t actually see any benefits to removing these existing types other than purity of design.

Target-specific hooks

In the previous discussion, it was noted that different targets may have different requirements for opaque types. For example, (this is my understanding) WebAssembly would like to have its opaque types renamed when linking in different modules (i.e., opaque(“wasm.gc”, i32 0) in one module isn’t necessarily the same as opaque(“wasm.gc”, i32 0) in another module). If x86_mmx were opaque(“x86.mmx”) instead, it would need to have a different size than the nominal sizeof(ptr) that I’ve given it here.

At present, the only real facility we have for indicating target-specific details of IR is the datalayout string. However, the datalayout string essentially requires listing out the properties of every possible type on the target for completeness, and there can be a very large number of rarely-used types. Indeed, one previous idea for representing these types were as non-integral address spaces, and the number of needed address spaces was a major factor in rejecting that idea. As a result, I don’t think it makes sense to shoehorn them in the datalayout string.

One possibility for specifying this information would be to embed it in the IR file when the opaque type is first used. For example, (assuming you’ve got something like a named opaque type):

%imgf2d = opaque(“image”, float, i32 2, i32 42)
%x86_mmx = bitcastable size(8) opaque(“image”)
%pipe = canbeglobal opaque(“pipe”)
%wasmgc = nominal opaque(“wasm.gc.foobar”)

This approach does pose some more challenging questions for how it interacts with linking LLVM modules together, which is again why I haven’t attempted to implement anything along these lines. (Also, working out what the set of properties would need to be would require more use cases and examples than I alone could provide).

Another approach is to extend target information beyond the existing TargetTransformInfo class to include things like TargetIRVerifier or TargetLlvmLinker that would better allow fuller target-specific modifications to the IR. Adding such classes could have ancillary benefits, for example, being able to identify types that cannot be supported on particular architectures (e.g., x86_fp80 or ppc_fp128), or opening up other paths to fixing calling convention issues at the LLVM IR level.

Instruction selection support

I don’t have much familiarity with the codegen section of LLVM, and my immediate use cases rely on direct lowering of LLVM IR to target-specific details without going through SelectionDAG or GlobalISel. As a result, I don’t know what needs to be changed in MVT or EVT to support opaque types.

Can values of this opaque type hold pointers?

If so, things are trickier because ‘icmp eq’ doesn’t imply value equivalence anymore.
E.g., it would be wrong for GVN to replace if (a == b) use(a); with if (a == b) use(b);

Thus it makes a difference whether we want a truly opaque thing that can hold anything, or limited (no pointers, for example).

I’m not entirely clear as to what you mean here—your example doesn’t indicate what the types of a and b are.

icmp eq is not a legal operation for opaque types. Integers and pointers, the things which can be used in icmp eq, can only be related to opaque types is via some sort of intrinsic call that uses them as parameters—at which point the semantics would be as normal (integers do not have provenance, pointers do), and I don’t see how opaque types change the situation.

Thanks for clarifying that icmp is not a valid operation. It wasn’t clear for me.

Another thing: for the sake of optimizations, it would be important to define that type punning can’t happen through memory, e.g., store a pointer, read an opaque type, pass it to an intrinsic and get the pointer back.
I guess you want to have AA completely ignore opaque types.

The semantics here feel a little weird. It’s sized like a pointer, it can be loaded and stored like a pointer, but it… somehow isn’t a pointer. I guess there’s some sort of hidden provenance, similar to pointer provenance, that gets destroyed if you convert a value? Or are you just making them sized types as a shortcut to avoid issues defining alloca etc. for unsized values?

x86_mmx isn’t really a good comparison point for anything; the representation broken at a fundamental level. See [X86] Add pass to insert EMMS/FEMMS instructions to separate MMX and X87 states · Issue #41664 · llvm/llvm-project · GitHub etc.

The primary reason for giving opaque types a size is that it makes it possible to use them in alloc/load/store, and I suspect it is lower burden on optimization passes to assume that opaque types have a size rather than to introduce non-sized allocas/loads/stores.

As for provenance, opaque types have similar provenance concerns to pointers, although unlike pointers, there is no ptrtoint/inttoptr escape hatch—they’re like nonintegral address space pointers in that regard.

Thank you for making this explicit proposal which I like quite a bit.

About the provenance concerns: I expect the use cases we care about to just work if we say that opaque-typed values carry no pointer provenance. The theoretical justification is that pointer provenance is primarily required to reason about the pointer arguments of load and store, where opaque values cannot be used.

The @evil example is affected because it essentially bitcasts from opaque(..) to ptr through memory. By saying that an opaque value has no provenance, this behaves like a bitcast from an integer to a pointer in terms of provenance. I’m not aware of a use case that would want to allow this bitcast in a way where the resulting pointer obtains provenance in some other, well-defined way.

If we ever do need opaque values with pointer provenance, that could become a type attribute, something like:

%type_with_provenance = opaque(..; has_provenance)

About the size: Picking “pointer of address space 0” as the size feels quite arbitrary. I understand that a size is required in practice to make load/store work. How about picking one of: 0; 1; always require an explicitly given size?

About naming and linking: The key question is whether opaque types are structural or not.

The GPU use cases I’m familiar with all want structural types. It seems like WASM my optionally want non-structural opaque types. Non-structural opaque types require naming, while for structural types it is convenient but may not be strictly necessary.

We can support both:

%spirv.queue = opaque("spirv.Queue")
%spirv.queue2 = opaque("spirv.Queue")

# bar has three arguments that all have the same type
declare void @bar(opaque("spirv.Queue"), %spirv.queue, %spirv.queue2)

# The following is an error because it attempts to define a structural opaque type
# whose structure is equal to an already existing structural type with different attributes.
declare void @bad(opaque("spirv.Queue"; size(1))

%nonstructural = newtype opaque("wasm.foo")
%nonstructural2 = newtype opaque("wasm.foo")

# baz has two arguments of different type
declare void @baz(%nonstructural, %nonstructural2)

# This syntax is forbidden because there's no way to define or use @quux:
declare void @quux(newtype opaque("..."))

Linking behavior falls out quite naturally:

  • structural types from different modules are equal based on their structure
  • non-structural (“newtype”) types from different modules are never equal

The scope of the @bad error in the example above is an LLVMContext. That is, the structure of structural opaque types is determined by their name and parameters, and attempting to create a second type with the same structure but different attributes in the same LLVMContext is an error. This implicitly answers the corresponding linker question (loading a module with conflicting attributes on an opaque type fails and therefore linking fails).

I don’t know if the above satisfies WASM’s requirements. An initial implementation could only support structural types and be extended later.

Syntax bikeshed: I slightly prefer putting any “type attributes” inside clear syntactic grouping. Compare:

  1. opaque("mytype", i32, i32 5; size(4) has_provenance)
  2. size(4) has_provenance opaque("mytype", i32, i32 5)
  3. opaque("mytype", i32, i32 5) size(4) has_provenance

I prefer the first option. It is one character longer but avoids parsing ambiguities. The third option is already problematic today because opaque type attributes may become ambiguous with function argument attributes, as in:

declare void @foo(opaque("mytype", i32, i32 5) size(4) has_provenance byval)

The second option may be okay today but risks future ambiguities.

Picking “pointer of address space 0” as the size feels quite arbitrary. I understand that a size is required in practice to make load/store work. How about picking one of: 0; 1; always require an explicitly given size?

The rationale pointer-sized simply because the most natural “real” lowering of an opaque type is a pointer (this is not always the case, in my understanding, but it seems reasonable in many cases). For example, Clang already decides that sizeof(opencl_*_t) is the same as sizeof(void*).

The theoretical justification is that pointer provenance is primarily required to reason about the pointer arguments of load and store, where opaque values cannot be used.

On further thought, I think provenance is entirely irrelevant to opaque types. The gist of provenance is that it means that values that compare equal may not be substitutable for one another, but we have no oracle that allows us to determine when two opaque types are equal in the first place, as converting an opaque type to a type that can be used in an icmp instruction is illegal. It’s possible that target intrinsics may permit such conversions to occur with intrinsics, but then it’s the target’s responsibility to document the resulting behavior.

The key question is whether opaque types are structural or not.

I’ll let @asb chime in here to comment on these portion, because he’s the one who best understands the constraints of WASM.

The one comment I will make right now is that I would prefer to avoid giving opaque types seemingly two names, as %spirv.queue = opaque("spirv.Queue") would do. If that is necessary to achieve functionality, then it is necessary to do, but if use cases can cleanly avoid it, I think that is preferable.

The scope of the @bad error in the example above is an LLVMContext. That is, the structure of structural opaque types is determined by their name and parameters, and attempting to create a second type with the same structure but different attributes in the same LLVMContext is an error. This implicitly answers the corresponding linker question (loading a module with conflicting attributes on an opaque type fails and therefore linking fails).

While I do agree that giving different attributes to the same opaque type should be an error, it gives me some pause in that it potentially creates a multiple-sources-of-truth issue. At this point, I’m leaning towards deferring the issue until there’s a compelling use case that calls for type attributes.

Why not include the representation, be it ptr, ptr addrspace(42), i64 or i51 in the opaque type definition itself? That wouldn’t carry any semantic meaning beyond “here’s what type of storage to use”.

1 Like

Part of the point of opaque type is that there isn’t a representation in terms of other types. In the example of SPIR-V image types, presumably a concrete underlying hardware implementation has some underlying type, but the SPIR-V backend as I understand it potentially targets vendor-agnostic SPIR-V.

One thing that occurred to me is that ScalableVectorTypes (e.g. <vscale x i32>) don’t have a size but can be used with alloca, load, store. Given that, perhaps having opaque types without a default size isn’t too onerous after all?

I’ll have to do so some experimentation to see how scalable vectors work without sizes. If we can get away without a size, that’s probably for the best.

Scalable vectors are still considered sized types: Their size is a scalable TypeSize of the form vscale x N. I don’t think there is any easy way to generalize this to a completely unknown size.

I’ve gone ahead and started a patch for review that adds the opaque type here: ⚙ D135202 [IR] Add an opaque type to LLVM.. I’ve tried to incorporate all of the feedback provided in the thread so far, but I may have missed something.

The patch doesn’t add any actual uses of opaque type yet–I can provide a patch that implements the SPIR-V special types as opaque types, but that isn’t polished enough yet for review. But this patch does have all the LLVM features I’ve found necessary and the corresponding documentation I could think to add. In particular, I haven’t tried to add any type attributes to opaque types as there hasn’t been a need to do so for my use-case. If this is a problem, I can adjust this patch to provide such attributes.

I would especially appreciate anyone representing other backends interested in using these types to comment on whether or not the current design suffices for their needs, and if it does not, to suggest what additions I need to make.

Sorry I’ve been slow in feeding back. I’ll study carefully and come back to this thread in the next few days.

Thanks for this. I’ve been quickly looking at the review and noticed the initial solution for the size of opaque types is to use the size of pointers in address space 0.

Going back to this discussion I’ve noticed that there has been some discussion about this:

About the size of OpenCL specific types in Clang.
Based on the code in https://github.com/llvm/llvm-project/blob/main/clang/lib/AST/ASTContext.cpp#L2205-L2208 it seems that the target controls the mapping between classes of OpenCL specific types to address spaces, as well as the size of pointers for a given address space.

In principle it is possible to have a target where different OpenCL specific types have different sizes.
Therefore it seems important to allow more flexibility w.r.t. the size of the opaque types.

Since the post of [RFC] Proposal for TLX: Tensor LLVM eXtensions is too old with less talk, I refer it here.

I think opaque type may be a good uniform way to handle different high level target types we need such as matrix, x86mmx. If we can’t handle those target type in one uniform solution, it will introduce many fragmentation code to handle different target type. Now there is already much x86-specific type and some are also on the fly, ⚙ D136861 [IR] Add LLVM IR support for aarch64_svcount opaque type.. So I think we urgently need consensus about how to handle high level type in LLVM IR.

How does backend handle target extension type? How to write codegen pattern in tablegen infra? I think before we add and propose something in LLVM IR we need consider middle-end and back-end together well.

As I am working on scalable matrix, I will give you some feedback later in using opaque type on whether or not the current design suffices my need. If I don’t understand wrong, the opaque type is like a type wrapper of underlying layout type. The size and alignment is getting from existing underlying type. We can wrap a struct, array or vector type as target type to make it structure and valid in register as single value type. So I think a wrap type to scalable vector type can workable for scalable matrix type.

I’ve been looking to use this for Wasm GC types. The representation I’m going with involves inlining type definitions that refer to each other (which should be fine for the structural types we’re using at the minute). e.g. the second type in

(type $bvec (array i8))
(type $vec (array (ref $bvec)))

would be written as target("(array (ref (array i8)))").

The main blocker I see right now is the limitation that disallows bitcasting. As our types are essentially user-defined, it’s not feasible to provide a complete set of intrinsics covering all the opaque types we’re likely to see (e.g. a struct.get which returns the value of the nth field in a given struct). The two options that come to mind would be:

  1. Some kind of ambitious expansion of intrinsics, allowing intrinsics to be type-parameterised (both in terms of the return type and arguments).
  2. Removing the restriction on bitcasting, allowing a placeholder opaque type to be used in these “generic” intrinsics and the frontend to emit appropriate casts for the arguments / returns. If a backend doesn’t know what to do with such bitcasts, it can error out.

Option 2) of course feels like the path of least resistance to me, and doesn’t feel like it should have a serious adverse impact on other opaque types users. What do you think? Or am I missing another path forwards?

When I implemented this type, I ultimately went with a per-type TargetExtType::Property bitmask approach, such that different target extension types can have different properties. Adding a Bitcastable property, and enabling that for the Wasm GC types, sounds like it would satisfy the needs here.

I’m probably misunderstand something here, but aren’t intrinsics already type parameterized? They usually have some kind of constraint, but don’t have to, e.g. ssa.copy works on arbitrary types.