[DebugInfo] Missing debug location updates for newly created instructions

IR & Optimizations
1

Hi,
While seeing newly created instructions with proper debug location updates, I have found some cases that no debug location updates for newly created instructions, even if they replace some old instructions in the IR programs. For example, IndVarSimplify-L368, -L394, JumpThreading-L2983, -L3120, -L1282.

Here is JumpThreading-L2983:

    PHINode *NewPN = PHINode::Create(SI->getType(), 2, "", SI->getIterator());
    NewPN->addIncoming(SI->getTrueValue(), Term->getParent());
    NewPN->addIncoming(SI->getFalseValue(), BB);
    SI->replaceAllUsesWith(NewPN);
    SI->eraseFromParent();

For these cases, although they may not cause confusing debugging behavior, I think we should give them proper debug location updates, avoiding potential debug info loss and improving the robustness of debug info.

However, I’m not sure whether these missing updates are intended, or the updates are necessary for newly created instructions, so I start this discussion, hoping to confirm my thought.

2 Likes
2

There are many cases where debug locations are dropped or lost during transformations. I believe @StephenTozer has started looking into this recently.

There might be existing issues filed for cases like this; or, there might not. If you would like to file issues, I recommend finding existing functional tests for these passes, and try using -debugify to find reproduceable examples.

3

If you would like to file issues, I recommend finding existing functional tests for these passes, and try using -debugify to find reproduceable examples.

I’ve tried -debugify. It reports warnings like “missing line xxx”, but how could I know which line is actually the missing line caused by missing the corresponding debug location update, instead of cases like “optimized out”? Or maybe I do not use -debugify in a proper way?

4

Debugify is generally used to detect cases where debug info has gone missing, though it also picks up a lot of false positives - @ntesic made some effort to remove those false positives a while back, but it’s not in trunk right now. In any case, it doesn’t have a way of generating a test case from a known bug.

What I’d recommend for generating a test case where you know the source code responsible would be to add an assert at the point of the broken code, either just assert(false) or using any necessary preconditions for the bug (e.g. assert(SI->getDebugLoc()) in this case, to ensure only cases where there’s an existing debug loc get detected), and then just compile some test programs until the assert is hit - a build of clang itself or of the LLVM test suite usually works. Once you’ve hit that error, you can use creduce or llvm-reduce to reduce the input until you have your reduced test case.

More broadly regarding missing line numbers, my brief dip into this issue indicates that there are a lot of places where line numbers are dropped unnecessarily, usually due to a new instruction being created to replace an old without the debug line being copied across; this is a fairly common pattern. I’ll hopefully put out a more comprehensive post later, but right now I believe that debug lines should probably be a required parameter for creating instructions - if an instruction is going to have no debug line, it should be explicitly stated rather than the default.

1 Like
5

Heh. Anti-pattern IMO.

Another tactic, more manual but probably faster, would be to take tests for the pass in question, add !dbg annotations manually to all instructions, then see whether the result of the pass leaves you with any new/modified instructions without those annotations. That’s likely a bug of the kind you are looking for.

6

right now I believe that debug lines should probably be a required parameter for creating instructions - if an instruction is going to have no debug line, it should be explicitly stated rather than the default.

+1

Missing debug information due to such cases can lead to performance loss when using AutoFDO which uses debug info to map hardware samples to source locations. At Google, we have been discussing recently how to tackle this in a systematic manner.

cc: @rnk @mtrofin

3 Likes
7

Related - Debugify was brought up there, see the comments around friction. Also see the last few comments - tl;dr; having a specific build bot where debugify was enabled would remove that friction. Happy to help with bot set up if that’s a blocker (esp. under the assumption it would also help the MD_prof case in the referenced RFC)

1 Like
8

What I’d recommend for generating a test case where you know the source code responsible would be to add an assert at the point of the broken code, either just assert(false) or using any necessary preconditions for the bug (e.g. assert(SI->getDebugLoc()) in this case, to ensure only cases where there’s an existing debug loc get detected), and then just compile some test programs until the assert is hit - a build of clang itself or of the LLVM test suite usually works. Once you’ve hit that error, you can use creduce or llvm-reduce to reduce the input until you have your reduced test case.

Yeah, I have used assert(false) to get the corresponding regression tests being processed by the instruction transformation without debug location updates. By this, I can understand the transformation straightforwardly and then construct a small C program representing the reduced testcase to check the debug location loss.

Right now, I’m working on automatically detecting missing debug location updates in LLVM Transformation source code (not -debugify), and many missing updates have been found. If the community is willing to accept the patches, I’d like to collect and patch them.

2 Likes
9

I’ve been thinking of moving in a similar direction (adding the capacity to detect not only the presence of missing lines, but the position in the LLVM source where they were lost), so I’d be interested to hear how you’re going about this. It’s something that I’d assume is easier to do if every instruction creation site has an explicit DebugLoc, hence would benefit from making DebugLoc a required parameter?

10

I’ve been thinking of moving in a similar direction (adding the capacity to detect not only the presence of missing lines, but the position in the LLVM source where they were lost), so I’d be interested to hear how you’re going about this.

According to what I learned from historical debugloc fixes and experience obtained when fixing previous unknown debug location update bugs, I summaried update rules to correctly update debug locations during the code transformation. Now, I’m trying to detect debug location errors by performing non-intrusive static analysis on the transformation code (in the form of LLVM IR), utilizing the summaried rules. However, only simple missing or incorrect preserving updates (eg., the reported missing updates) can be detected now.

It’s something that I’d assume is easier to do if every instruction creation site has an explicit DebugLoc , hence would benefit from making DebugLoc a required parameter?

Explicitly assigning a DebugLoc at every instruction creation site sounds like a good solution to eliminating the missing updates of debug locations of newly created instructions. But it seems that implementing this needs great efforts. BTW, from the fixes I collected, most of the update errors are exists in “old code” (updates are properly handled in code added recently)?.

11

That might be the case, but in some experiments I ran I found that over time (from LLVM-10 to 18), the availability of source lines in -O2 -g builds with LLVM has decreased, measured as “number of source lines with at least one associated instruction”; this might just result from more aggressive optimization that legitimately removes debug locs, but it means it’s probably worth scrutinizing newer code (and possibly disabling some of those optimizations in the proposed “optimize for debug info” optimization flags).

1 Like