[HLSL][SPIRV] NSDI debug info support for clang-dxc

Clang Frontend
, ,
1

I’m working on implementing NonSemantic.Shader.DebugInfo (NSDI) debug info support for HLSL shaders compiled to SPIR-V via clang-dxc. In parallel, I am also extending NSDI to support new SPIR-V extensions and have recently unified its versioning scheme, so we can keep adding new instructions for any SPIR-V extensions that may need new debug instruction support. Additionally, I want to make sure that clang-dxc supports emitting versioned NSDI.

I have been learning how Clang and the SPIR-V backend deal with debug info and I have some ideas and questions that I would like to discuss here. I’m new to the SPIR-V backend, so some of what I say below may be wrong, and I’m looking for corrections. My goal is to understand the design well enough to contribute the HLSL-specific pieces without getting in the way.

Current state of the NSDI pass

The LLVM SPIR-V backend uses SPIRVEmitNonSemanticDI to emit DebugSource, DebugCompilationUnit, DebugTypeBasic, and DebugTypePointer. Nothing is emitted per-function: no function extents, no line numbers, no variable locations.

I found three active PRs extending this:

  1. #183117: Refactors the pass to add CompileUnitRegMap and per-function DI emission.
  2. #183121: Fixes SPIRVModuleAnalysis to use a blacklist for routing NSDI instructions to the correct layout section.
  3. #183122 (WIP): Adds DebugFunction, DebugTypeFunction, and DebugFunctionDefinition.

There is also #179975, a WIP PR covering the complete NSDI instruction set and a ModulePass conversion. @mgcarrasco, I understand that you are breaking it up into smaller pieces?

Beyond the general NSDI backend work, I found some HLSL-specific issues:

  1. GetSourceLanguage() does not check LO.HLSL, so HLSL shaders compiled with -g get tagged with DW_LANG_C_plus_plus_14 instead of DW_LANG_HLSL. The NSDI pass correctly maps DW_LANG_HLSL to language code 5, but the mapping is never triggered. This relates to issues #136929 and #136995.
  2. HLSL vectors (float4, int3) are lowered to llvm::FixedVectorType before debug info is emitted. The debug metadata contains an anonymous array type rather than the HLSL vector name. NSDI has DebugTypeVector (opcode 6) for this case.
  3. Same issue for HLSL matrices (float4x4). NSDI has DebugTypeMatrix (opcode 108).
  4. HLSLAttributedResourceType::CreateType() delegates to the wrapped type. RWBuffer<float> appears as float in debug info. The DXIL path was fixed in #119041; the SPIR-V path was not.
  5. No DebugEntryPoint (opcode 107) emission for HLSL shader entry points.
  6. The NSDI pass is not activated automatically when -g is passed to a SPIR-V target. For clang-dxc, it should be.

There may be others. I’ve only just begun looking at this.

Design direction

AFAIU, LLVM has two debug writers: DwarfDebug and CodeViewDebug. They target different output formats but share the same architecture. Both implement the AsmPrinterHandler interface and are registered with AsmPrinter as debug handlers. The shared lifecycle is: beginModule collects compile units from llvm.dbg.cu and emits module-scope singletons; beginFunction / endFunction process per-function metadata; endModule flushes deferred output. Type emission is lazy in both: types are lowered on first reference via getOrCreate* helpers, not pre-collected. Neither writer is a pass.

In my view, the NSDI writer should work exactly the same way. The only difference is the output format: it emits OpExtInst NonSemantic.Shader.DebugInfo instructions instead of DWARF DIEs or CodeView records. The IR metadata mapping is direct:

  • DICompileUnit becomes DebugCompilationUnit
  • DISubprogram becomes DebugFunction + DebugFunctionDefinition
  • DILocalVariable becomes DebugLocalVariable
  • DILocation becomes DebugLine
  • DIExpression becomes DebugExpression + DebugOperation

Concretely, I would replace SPIRVEmitNonSemanticDI by an AsmPrinterHandler subclass registered in SPIRVAsmPrinter, not extended as a pass. This is identical to how DwarfDebug works: it is a handler owned and driven by the printer, not a pass that runs alongside it. This way, we get:

  1. No deduplication logic. beginModule runs once. DebugCompilationUnit and all module-scope type instructions are singletons by construction. No tracking maps or dedup guards are needed.
  2. Correct placement by construction. Instructions emitted in beginModule precede any OpFunction. Instructions emitted in beginFunction stay in the function body. The placement routing in SPIRVModuleAnalysis becomes unnecessary for NSDI instructions.
  3. Lazy type deduplication. A getOrCreateDebugType(DIType *) helper emits each type once and returns its result register. Recursive types are handled via the same deferred-completion pattern CodeViewDebug uses.
  4. Topological ordering. Types emitted lazily on first reference produce a valid topological order automatically. No pre-pass to sort types is needed.
  5. Re-use existing support. AsmPrinterHandler subclasses have access to DbgValueHistoryMap (variable location tracking), DebugLocEntry (location lists), and DwarfExpression (DIExpression lowering). These cover the hardest parts of a debug writer. Reusing them avoids re-implementing variable location logic from scratch. This has been a difficult issue to fix in DXC because SPIRV-Tools optimizer does not handle debug info properly.

I’m not certain whether SPIRVAsmPrinter already supports addDebugHandler or whether that requires additional backend changes. I’m also not familiar enough with the SPIR-V backend’s instruction emission model to know whether an AsmPrinterHandler has direct access to the SPIR-V instruction builder at the points it needs it. These are constraints I may be underestimating, and I’m looking for input.

What I would suggest:

  1. Land PR #183121 now. It is independent from my concerns.
  2. Convert SPIRVEmitNonSemanticDI to an AsmPrinterHandler subclass registered in SPIRVAsmPrinter. This goes further than the ModulePass conversion in #179975 and aligns the NSDI writer structurally with DwarfDebug and CodeViewDebug.
  3. Re-layer the features from #183117 and #183122 on top of the handler.

I may be missing backend constraints that make the AsmPrinterHandler conversion impractical. If so, the ModulePass path from #179975 is a reasonable intermediate step. Either way, I don’t think the MachineFunctionPass model works well long-term.

My focus is on the HLSL-specific pieces that are not covered by the general NSDI backend work:

  1. Fix GetSourceLanguage() to check LO.HLSL before LO.CPlusPlus. This is a small standalone PR; it links to issues #136929 and #136995.
  2. Frontend representation for HLSL vectors and matrices that preserves the HLSL type name in DICompositeType, enabling correct DebugTypeVector and DebugTypeMatrix emission.
  3. DebugTypeMatrix (opcode 108) emission in SPIRVEmitNonSemanticDI.
  4. DebugEntryPoint (opcode 107) emission for HLSL entry points.
  5. Automatic pass activation when -g is passed to a SPIR-V target.

Questions

  1. @mgcarrasco: Is there a technical reason the MachineFunctionPass model has to be kept for #183117 and #183122, rather than converting SPIRVEmitNonSemanticDI to an AsmPrinterHandler subclass first? The AsmPrinterHandler lifecycle (beginModule / beginFunction / endFunction / endModule) maps directly onto NSDI’s module-scope vs. function-scope split, and the features in both PRs can be ported to it without deduplication logic. If there are backend constraints that make this impractical, I’d like to understand them, since they affect how the HLSL-specific contributions should be structured.
  2. Would it be useful to split just the SPV_KHR_non_semantic_info OpExtension declaration out of #179975 into a minimal standalone PR?
  3. Is there someone working on DebugLine / DebugNoLine (opcodes 103, 104)? The prior attempt (#113541) identified two problems that are now addressable. I’m happy to contribute this if it is not already claimed.

@mgcarrasco, @s-perron, @Keenuts, @beanz, @echristo, @dblaikie, and anyone else interested in SPIR-V debug info support: does this sound reasonable? Please correct me on any misunderstandings I may have about how debug info emission should work.

Finally, what is the right forum to discuss design ideas? Here on discourse? As a github issue on llvm-project?

Thanks. Diego.

1 Like
2

@dnovillo thanks for sharing this. It is super insightful. I’m new to debug info and SPIR-V so this has helped.

In summary, your idea sounds great to me if we can make it work. My goal is to eventually support NonSemantic.Shader.DebugInfo.100 and also make sure that it works well with the SPIRV-LLVM translator. Hopefully, here are some answers to the questions plus some new questions from my side.

Is there a technical reason the MachineFunctionPass model has to be kept for #183117 and #183122, rather than converting SPIRVEmitNonSemanticDI to an AsmPrinterHandler subclass first?

Regarding #183117 and #183122, I wasn’t aware of AsmPrinterHandler. Your plan sounds great to me if it can fit in the current backend’s implementation.

The PRs kept the MachineFunctionPass model because my understanding is that the BE has no notion of SPIR-V module until the end. For example, global variable are affected in a similar way, meaning that each function declares its own copy until the SPIR-V module is materialized and deduplication is applied. I was planning to do similarly for DebugCompilationUnit and DebugFunction. However, I like your plan because this may be avoided (and also for other opcodes).

I’m unsure whether it is safe in the context of SPIR-V BE to refer to registers across different function definitions. The AsmPrinterHandler approach may not face this problem because it is triggered at the time that the SPIR-V module is materialized.

I’m trying to picture possible challenges based on my SPIR-V/BE understanding. I think you may have already considered this but I’m raising this up just to be safe.

The main challenge that I see is that some instructions must be emitted in specific sections. If I follow, the AsmPrinterHandler approach must be aware of it. In other words, the AsmPrinterHandler must make sure that when emitting an instruction all its required operands, if they belong to a different section, they are already emitted (and keep track of their registers), right?

Could this approach still have the risk of emitting duplicate instructions? If so, can it be mitigated?

Is there someone working on DebugLine / DebugNoLine (opcodes 103, 104)?

Last time I asked in the LLVM SPIRV Backend meeting no one else was working on debug info. My original plan was to tackle those instructions once we had support for DISubProgram and DebugFunction, although I’d be more than happy to coordinate our efforts.

There is also #179975, a WIP PR covering the complete NSDI instruction set and a ModulePass conversion. @mgcarrasco, I understand that you are breaking it up into smaller pieces?

I tried to break that specific PR into smaller pieces but found too many issues at once and its size didn’t help either. In short, most of the its tests were not reversed translatable by the SPIRV-LLVM translator. I recall that at least it was somehow breaking the debug info support that is already in LLVM upstream.

I’ve been working on submitting smaller PRs on the same direction also to help in the reviewing process. #183117 and 183122 are my attempts so far. I wanted to gather feedback as soon as possible in case I was missing things like the ones you pointed out (thanks).

Concretely, I would replace SPIRVEmitNonSemanticDI by an AsmPrinterHandler subclass registered in SPIRVAsmPrinter, not extended as a pass.
Convert SPIRVEmitNonSemanticDI to an AsmPrinterHandler subclass registered in SPIRVAsmPrinter. This goes further than the ModulePass conversion in #179975 and aligns the NSDI writer structurally with DwarfDebug and CodeViewDebug.

This sounds good to me. If a PR is submitted just for the replacement/switch I’d be happy to review it. I can also relayer #183117 and #183122 and work on extra steps towards NSDI support within this new approach.

Land PR #183121 now. It is independent from my concerns.

If I follow, the NSDI writer approach would already emit the instructions in the appropriate sections and the handling in #183121 may no longer be required, right?

Questions:

  • Just to learn, I noticed that you mentioned specializing the AsmPrinterHandler class but not the DebugHandlerBase one. Is there any reason for this?

Remarks:

  • For us it is important to make sure that the emitted SPIR-V code can also be reversed translatable using the SPIRV-LLVM translator, although I know this somewhat orthogonal to the approach used to emit debug info in the BE.

Once again thanks for sharing this idea!

Best,
Manuel

3

Hello!

Thanks for the detailed writeup!
Overall, seems coherent, and I don’t see any large issues (also not super familiar with the debug info code part)

There is one nit here: when emitting debug instructions for class/objects, there is a circular dependencies on IDs: DebugTypeComposite declares the function, but the function declares the composite as a scope (because of this pointer).
In this specific case you need to emit SPV_KHR_relaxed_extended_instruction instead. Otherwise should not cause ordering issues.

4

@dnovillo thanks for sharing this. It is super insightful. I’m new to debug info and SPIR-V so this has helped.

Thanks.

In summary, your idea sounds great to me if we can make it work. My goal is to eventually support NonSemantic.Shader.DebugInfo.100

Soon to be NSDI.101 :wink:

The PRs kept the MachineFunctionPass model because my understanding is that the BE has no notion of SPIR-V module until the end. […] I’m unsure whether it is safe in the context of SPIR-V BE to refer to registers across different function definitions. The AsmPrinterHandler approach may not face this problem because it is triggered at the time that the SPIR-V module is materialized.

Right. AsmPrinterHandler runs during emission, after all MI passes have completed and the SPIR-V module is fully materialized. The per-function copy and late deduplication issue applies to the pass pipeline, not to the emitter. Module-scope debug instructions emitted in beginModule are assigned IDs before any OpFunction is emitted, so function-scope instructions can safely reference them.

[ @echristo @dblaikie please correct me if I’m babbling nonsense here ]

The main challenge that I see is that some instructions must be emitted in specific sections. If I follow, the AsmPrinterHandler approach must be aware of it. In other words, the AsmPrinterHandler must make sure that when emitting an instruction all its required operands, if they belong to a different section, they are already emitted (and keep track of their registers), right?

Yes, the handler must emit into the correct logical layout sections. The advantage over the pass approach is that the handler controls when it emits relative to the module structure, so placement is correct by construction rather than requiring post-processing.

Could this approach still have the risk of emitting duplicate instructions? If so, can it be mitigated?

I don’t think this can happen, actually. beginModule runs once. Module-scope singletons like DebugCompilationUnit are emitted there and nowhere else. Type instructions are guarded by a getOrCreateDebugType helper that emits each type on first reference only. We should not need to have any tracking or deduplication anywhere.

If I follow, the NSDI writer approach would already emit the instructions in the appropriate sections and the handling in #183121 may no longer be required, right?

Yeah, I think so. From what I read, #183121 fixes NSDI instruction routing in SPIRVModuleAnalysis because the pass-based approach cannot control section placement directly (did I read it right?). If the handler controls placement by construction, we won’t need it.

Just to learn, I noticed that you mentioned specializing the AsmPrinterHandler class but not the DebugHandlerBase one. Is there any reason for this?

Sorry, yeah, that was wrong. DebugHandlerBase is the one we need to sub-class.

For us it is important to make sure that the emitted SPIR-V code can also be reversed translatable using the SPIRV-LLVM translator, although I know this somewhat orthogonal to the approach used to emit debug info in the BE.

Right. We need to verify reverse translation as we add instructions.

Thanks. Diego.

1 Like
5

There is one nit here: when emitting debug instructions for class/objects, there is a circular dependencies on IDs: DebugTypeComposite declares the function, but the function declares the composite as a scope (because of this pointer). In this specific case you need to emit SPV_KHR_relaxed_extended_instruction instead.

Ah, yes, thanks. Let me make sure I am following you: DebugTypeComposite for a class lists its member functions as members. Each member function’s DebugFunction takes the composite as its scope because of the this pointer. That is a forward reference that OpExtInst cannot express directly, since operand IDs must be defined before the instruction that uses them.

We then need to emit OpExtInstWithForwardRefsKHR for any NSDI instruction that has a forward reference operand.

Are there other known cases of circular dependency in NSDI beyond this?

Thanks. Diego.

6

Yes your understanding is correct. As far as I know, that only happens with objects, don’t have others top of mind.

7

AFAIU, LLVM has two debug writers: DwarfDebug and CodeViewDebug. They target different output formats but share the same architecture. Both implement the AsmPrinterHandler interface and are registered with AsmPrinter as debug handlers. The shared lifecycle is: beginModule collects compile units from [llvm.dbg.cu](http://llvm.dbg.cu) and emits module-scope singletons; beginFunction / endFunction process per-function metadata; endModule flushes deferred output. Type emission is lazy in both: types are lowered on first reference via getOrCreate* helpers, not pre-collected. Neither writer is a pass.

In my view, the NSDI writer should work exactly the same way.

@mgcarrasco, @s-perron, @Keenuts, @beanz, @echristo, @dblaikie, and anyone else interested in SPIR-V debug info support: does this sound reasonable? Please correct me on any misunderstandings I may have about how debug info emission should work.

Finally, what is the right forum to discuss design ideas? Here on discourse? As a github issue on llvm-project?

This all works for me. I’ll let the spir-v folks comment on the specific plan to move things over, but in general I think it can work. I’m uncertain whether or not we want it to live in the spir-v backend or in the asm printer, but I’m inclined to say the latter since it’s a different style of debug info even if it’s somewhat backend tied.

Thoughts?

-eric

8

This all works for me. I’ll let the spir-v folks comment on the specific plan to move things over, but in general I think it can work. I’m uncertain whether or not we want it to live in the spir-v backend or in the asm printer, but I’m inclined to say the latter since it’s a different style of debug info even if it’s somewhat backend tied.

Thanks. Ideally, I’d like to treat NSDI as another debug format that asm printer can handle. The fact that NSDI is heavily inspired on DWARF may make it straightforward to implement.

I think right now we could consider it tied to a single backend simply because it’s the only backend that consumes NSDI. In my view, it’s akin to having DWARF tied to the x86 backend because that’s the only backend that understands DWARF (yeah, I’m stretching it…)

Diego.

9

At a high level, it seems reasonable to me. I like anything that gets the SPIR-V backend to use more standard LLVM solutions to problems.

10

Hi,

Thanks for opening this discussion, it sounds good that your debug-info format fits into LLVMs existing DebugHandler model.

I know very little about HLSL, SPIRV, or anything in the shader space, but would there be any high level documentation of this new format (NSDI) and what its objectives are, and why existing DWARF etc isn’t suitable? I’m interested in the possibility that it doesn’t fully overlap with our existing debug-info emitters, meaning we’d have to add more plumbing to support it – which is probably fine, but best to know early.

Thanks,

Jeremy

11

Yes, the official spec for NSDI is at SPIR-V NonSemantic Shader DebugInfo Instructions. Since this debug info needs to live inside a SPIR-V shader module, the debug info format needs to fit within the framework defined by non-semantic extended instruction sets.

When the SPIR-V module is translated to ISA, the mapping from NSDI to ISA can take the form of DWARF and/or PDB. You’ll see that the NSDI spec took a lot of inspiration from DWARF.

I have the vague suspicion that it will be fairly straightforward, but I don’t have good knowledge yet. As I work through the plan, I will make every attempt to not require changes in the plumbing for debug-info emitters.

Thanks. Diego.