What does it mean by a "DICompileUnit missing from llvm.dgs.cu" ? | Where is the error ?

Hello,

One of my Julia programs failed to execute with the following error,
DICompileUnit not listed in llvm.dbg.cu
!11 = distinct !DICompileUnit(language: DW_LANG_C89, file: !3, producer: “julia”, isOptimized: true, runtimeVersion: 0,emissionKind: FullDebug, enums: !12)

This was generated by verifyModule(*GPUModule,&(llvm::errs())From whithin PPCGCodeGeneration.cpp in Polly.

From <llvm_src>/test/Verifier/dbg-orphaned-compileunit.ll, I understand that the debug info, of a compile unit, represented by !11 is missing from llvm.dbg.cu.

What does this mean ? What could have caused this problem and how can I fix it ?

Thank You,
Sanjay

Hello,

One of my Julia programs failed to execute with the following error,
DICompileUnit not listed in llvm.dbg.cu
!11 = distinct !DICompileUnit(language: DW_LANG_C89, file: !3, producer: “julia”, isOptimized: true, runtimeVersion: 0,emissionKind: FullDebug, enums: !12)

This was generated by verifyModule(*GPUModule,&(llvm::errs())From whithin PPCGCodeGeneration.cpp in Polly.

From <llvm_src>/test/Verifier/dbg-orphaned-compileunit.ll, I understand that the debug info, of a compile unit, represented by !11 is missing from llvm.dbg.cu.

What does this mean ? What could have caused this problem and how can I fix it ?

At the LLVM level, it means that the CU debug info (DICompileUnit) wasn’t listed in the list of all compile units ( global named metadata node) when it should be. Whatever produced the IR you are using likely has a bug and should be fixed to ensure the DICompileUnits created are included in the global named metadata node (check Clang’s behavior for comparison - usually a good reference point)

This invariant was added a while ago (in LLVM time - I think it was in the last year or maybe two) by Adrian.

  • Dave

> Hello,
>
> One of my Julia programs failed to execute with the following error,
> DICompileUnit not listed in llvm.dbg.cu
> !11 = distinct !DICompileUnit(language: DW_LANG_C89, file: !3, producer:
> "julia", isOptimized: true, runtimeVersion: 0,emissionKind: FullDebug,
> enums: !12)
>
> This was generated by verifyModule(*GPUModule,&(llvm::errs())From whithin
> PPCGCodeGeneration.cpp in Polly.
>
> From <llvm_src>/test/Verifier/dbg-orphaned-compileunit.ll, I understand
> that the debug info, of a compile unit, represented by !11 is missing from
> llvm.dbg.cu.
>
> What does this mean ? What could have caused this problem and how can I
> fix it ?
>

At the LLVM level, it means that the CU debug info (DICompileUnit) wasn't
listed in the list of all compile units (!llvm.dbg.cu global named
metadata
node) when it should be. Whatever produced the IR you are using likely
has
a bug and should be fixed to ensure the DICompileUnits created are
included
in the !llvm.dbg.cu global named metadata node (check Clang's behavior
for
comparison - usually a good reference point)

This invariant was added a while ago (in LLVM time - I think it was in
the
last year or maybe two) by Adrian.

Right. It seems as if Julia is emitting invalid debug symbols. If you
want to point them to this issue, you can probably call
"verifyModule(*GPUModule))" after their initial IR is generated and
already see the issue. If this is not the case, Polly might make a
mistake when extracting the GPU module and might miss to transfer all
debug information. Can you send me the output of the flag
"-polly-dump-before"?
Can you pass the dumped IR through 'opt' without seeing an error? In
this case, the
IR that enters Polly is still correct, but Polly might mis-transform it.

An alternative fix would be to extract the debug-info stripping
functionality in lib/Transforms/IPO/StripSymbols.cpp into its own
function and call this on our Kernel-IR. It is unlikely that we want to
debug GPU kernels, so we can just remove debug information for now.

Best,
Tobias

Hello,

I enabled “-polly-dump-before -polly-dump-after” to obtain the IR before and after Polly. Have attached kernel_gemm-before.ll (kg_before.ll) and kernel_gemm-after.ll (kg_after.ll)

observations,

  • Both kg_before.ll and kg_after.ll have,

  • The only DICompileUnit is at !2

  • =!{!2}

  • I don’t think Julia is generating incorrect debug info .

  • For opt kernel_gemm-before.ll $FLAG -o output.ll

  • All the following FLAGs except “-O3 -polly -polly-target=gpu” didn’t have a problem with the Module,

  • -O3 -polly -polly-target=cpu

  • -O3- The only time verifyModule is called within Polly is at kernerlFinalize.

  • This could mean that Polly’s generating the wrong debug info, which is uncovered only when GPU code’s generated.

  • I’m not sure if verifyModule is called indirectly when -polly-target=cpu

  • Or, the debug information is corrupted when sending or/and receiving information to/from PPCG.

  • For opt output.ll -O3 -polly -polly-target=gpu -o output.output.ll, where output.ll generated by opt kg_before.ll -O3 -polly -polly-target=cpu,

  • verifyModule is involed thrice, but issues the same error

  • Possibly 3 SCoPs detected ?- This increases the possibility that Polly isn’t handling the debug info properly

  • For opt kernel_gemm-after.ll $ANY_FLAG -o output.ll

  • Cannot invoke an intrinsic other than donothing, patchpoint, statepoint, coro_resume or coro_destroy

  • store void (metadata, i64, metadata, metadata)* @llvm.dbg.value, void (metadata, i64, metadata, metadata)** %polly_launch_0_param_7

  • Invalid user of intrinsic instruction!

  • store void (metadata, i64, metadata, metadata)* @llvm.dbg.value, void (metadata, i64, metadata, metadata)** %polly_launch_0_param_7

  • ./llvm_build/bin/opt: kernel_gemm-after.ll: error: input module is broken!

  • and output.ll isn’t produced.

-Sanjay

Only target=gpu giving me error, but verufyModule is only in the path of target=gpu.

kernel_gemm-before.ll (12 KB)

kernel_gemm-after.ll (30.3 KB)

Hello,

I enabled "-polly-dump-before -polly-dump-after" to obtain the IR before
and after Polly. Have attached kernel_gemm-before.ll (kg_before.ll) and
kernel_gemm-after.ll (kg_after.ll)

Perfect.

observations,

   - Both kg_before.ll and kg_after.ll have,
      - The only DICompileUnit is at !2
      - !llvm.dbg.cu=!{!2}
      - I don't think Julia is generating incorrect debug info .

Neither do I.

      - For opt kernel_gemm-before.ll $FLAG -o output.ll
      - All the following FLAGs except "-O3 -polly -polly-target=gpu"
      didn't have a problem with the Module,
         - -O3 -polly -polly-target=cpu
         - -O3

      - The only time verifyModule is called within Polly is at
      kernerlFinalize.
         - This could mean that Polly's generating the wrong debug info,
         which is uncovered only when GPU code's generated.
            - I'm not sure if verifyModule is called indirectly when
            -polly-target=cpu
            - Or, the debug information is corrupted when sending or/and
         receiving information to/from PPCG.

I think we are just generating incorrect code when extracting the GPU
module.

         - For opt output.ll -O3 -polly -polly-target=gpu -o
   output.output.ll, where output.ll generated by opt kg_before.ll -O3
   -polly
   -polly-target=cpu,
      - verifyModule is involed thrice, but issues the same error
      - Possibly 3 SCoPs detected ?
      - This increases the possibility that Polly isn't handling the
      debug
      info properly
      - For opt kernel_gemm-after.ll $ANY_FLAG -o output.ll
      - Cannot invoke an intrinsic other than donothing, patchpoint,
      statepoint, coro_resume or coro_destroy
      - store void (metadata, i64, metadata, metadata)*
      @llvm.dbg.value,
      void (metadata, i64, metadata, metadata)** %polly_launch_0_param_7
      - Invalid user of intrinsic instruction!
      - store void (metadata, i64, metadata, metadata)*
      @llvm.dbg.value,
      void (metadata, i64, metadata, metadata)** %polly_launch_0_param_7
      - ./llvm_build/bin/opt: kernel_gemm-after.ll: error: input module
      is
      broken!
      - and output.ll isn't produced.

The following patch highlights the first issue:

--- a/lib/CodeGen/PPCGCodeGeneration.cpp
+++ b/lib/CodeGen/PPCGCodeGeneration.cpp
@@ -1163,6 +1163,8 @@ GPUNodeBuilder::createLaunchParameters(ppcg_kernel
*Kernel, Function *F,
   }

   for (auto Val : SubtreeValues) {
+ errs() << "Referenced Parameter\n";
+ Val->dump();
     Instruction *Param = new AllocaInst(
         Val->getType(), Launch + "_param_" + std::to_string(Index),
         EntryBlock->getTerminator());
@@ -1254,8 +1256,8 @@ void GPUNodeBuilder::createKernel(__isl_take
isl_ast_node *KernelStmt) {
   Value *GridDimX, *GridDimY;
   std::tie(GridDimX, GridDimY) = getGridSizes(Kernel);

- createCallLaunchKernel(GPUKernel, GridDimX, GridDimY, BlockDimX,
BlockDimY,
- BlockDimZ, Parameters);
+ // createCallLaunchKernel(GPUKernel, GridDimX, GridDimY, BlockDimX,
BlockDimY,
+ // BlockDimZ, Parameters);
   createCallFreeKernel(GPUKernel);

opt -polly-codegen-ppcg kernel_gemm-before.ll

opt kernel_gemm-before.ll -polly-codegen-ppcg -polly-acc-dump-kernel-ir
WARNING: You're attempting to print out a bitcode file.
This is inadvisable as it may cause display problems. If
you REALLY want to taste LLVM bitcode first-hand, you
can force output with the `-f' option.

Referenced Parameter

; Function Attrs: nounwind readnone
declare void @llvm.dbg.value(metadata, i64, metadata, metadata) #2

Referenced Parameter
i64 %1
Referenced Parameter
i64 %0
Cannot invoke an intrinsic other than donothing, patchpoint, statepoint,
coro_resume or coro_destroy
  store void (metadata, i64, metadata, metadata)* @llvm.dbg.value, void
  (metadata, i64, metadata, metadata)** %polly_launch_0_param_7
LLVM ERROR: Broken function found, compilation aborted!

We try to pass the llvm.dbg.value function call to the kernel. This does
not work. We should not list @llvm.dbg.value in the Subtree values that
are passed to the kernel.
In fact, we should ignore all debug tree values and not code-generate
them at all.

Best,
Tobias