Questions on reproducibility/deterministic build

Hi,

I’m working on reproducibility (or deterministic build) issues with clang.

And I got issue with compiler-rt libraries, some of the debug symbols comes with source absolute path, e.g.:

strings ./build/lib64/linux/libclang_rt.fuzzer-x86_64.a | grep “lambda at”

—8<— A

CollectFeatures<(lambda at /compiler-rt/lib/fuzzer/FuzzerDriver.cpp:482:25)>

—8<— B

void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at

/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:77:29)]
void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at

/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:85:29)]
void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at

/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:99:29)]
void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at

/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:206:33)]
void fuzzer::TracePC::IterateCoveredFunctions(CallBack) [CallBack = (lambda at

/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:314:34)]

—8<— C
IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:77:29)>
IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:85:29)>
IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:99:29)>
IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:206:33)>
IterateCoveredFunctions<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:314:34)>

Conditions:

In section A and C debug symbols for lambda have path correctly remapped with prefix set in -f<file/debug/macro>-prefix-map.

But section B debug symbols for the same lambda code goes without remapping (and it’s ok TypePrinter code works properly in this case).

For section B TypePrinter::printTag called with PrintingPolicy.RemapFilePaths set, but for section A the same function called with different PrintingPolicy and RemapFilePaths is not set.

  • I’m not sure is it ok that two sections generated in different way and what is the reason for this? The similar code with lambdas from section A produce only one symbol type with remapped path.

  • Is it possible and is it make sense to pass CGDebugInfo settings to the code Sema::BuildPredefinedExpr to apply

RemapFilePaths settings to the PrintingPolicy used? As I understand symbols could be generated into different compilation stages/passes and some code generation settings could not be applied directly during other stages.

Regards,

Oleksiy

Could you provide a small standalone example and precise commands to reproduce the case you’re describing?

Hi David,

I have to notice that I tried to reproduce the same isue for clang-9.0 Zeus but without success.

So seems that it’s actual only for Thud clang-7.1.

I’m not sure if it worth doing but anyway here is how to reproduce:

I’m using standalone openembedded-core to reproduce.

diff --git a/recipes-devtools/clang/compiler-rt_git.bb b/recipes-devtools/clang/compiler-rt_git.bb
index 05de71f…b201814 100644
— a/recipes-devtools/clang/compiler-rt_git.bb
+++ b/recipes-devtools/clang/compiler-rt_git.bb
@@ -25,6 +25,8 @@ DEPENDS_append_class-nativesdk = " clang-native"
THUMB_TUNE_CCARGS = “”
#TUNE_CCARGS += “-nostdlib”

+DEBUG_PREFIX_MAP += " -fdebug-prefix-map=${S}=. "

0001-Honor-fdebug-prefix-map-when-creating-function-names.patch (4.47 KB)

Yeah, sorry - that's a bit too involved for me to get into at the
moment (& a somewhat older version of clang too - LLVM point releases
are generally only for the last release, not as far back as 7.0, so
even if we figured out what it was, if it doesn't reproduce in more
recent clang, there's probably nothing to be done here)