Weird behavior of llvm-ld

Bram_Adams · August 16, 2006, 4:01pm

Hi,

I’m using LLVM 1.8 with the GCC4-frontend on a Slackware 10.2 box (GCC 3.3.6). When issuing the following command (line 14 in a script called manual.sh):

llvm-ld -O1 *.trans -L${SOME_LIB_DIR} -L${SOME_OTHER_LIB_DIR} -L${LLVM_FRONT}/lib -lSOME_LIB -lSOME_OTHER_LIB -load=${MY_LIB_DIR}/opt1.so -load=${MY_LIB_DIR}/opt2.so -debug-pass=Details -debug -o test

… I get the following messages:

llvm-ld: warning: Supposed library ‘SOME_OTHER_LIB’ isn’t a library.
./manual.sh: line 14: 25492 Segmentation fault

When I invoke the same from within gdb, I get:

[Thread debugging using libthread_db enabled]
[New Thread -1210504992 (LWP 25520)]
llvm-ld: warning: Supposed library ‘SOME_OTHER_LIB’ isn’t a library.

Program received signal SIGABRT, Aborted.
[Switching to Thread -1210504992 (LWP 25520)]
0xb7dbb027 in raise () from /lib/tls/libc.so.6

… with the following backtrace:

#0 0xb7dbb027 in raise () from /lib/tls/libc.so.6
#1 0xb7dbc747 in abort () from /lib/tls/libc.so.6
#2 0xb7eb6721 in uw_init_context_1 () from ${LLVM_FRONT}/lib/libgcc_s.so.1
#3 0xbf9fd140 in ?? ()
#4 0xbf9fcf20 in ?? ()
#5 0x00000064 in ?? ()
#6 0x082a02cd in llvm::Optimize (M=0x85bf330) at ${LLVM_SRC}/tools/llvm-ld/Optimize.cpp:180
#7 0x082a7161 in main (argc=17, argv=0xbf9fd454, envp=0xbf9fd49c) at ${LLVM_SRC}/tools/llvm-ld/llvm-ld.cpp:467

The exception raised apparently was not caused by the first warning message, but resulted from the following snippet of ${LLVM_SRC}/tools/llvm-ld/Optimize.cpp (near line 180):

std::vectorstd::string plugins = LoadableModules;
for (std::vectorstd::string::iterator I = plugins.begin(), E = plugins.end(); I != E; ++I) {
sys::DynamicLibrary dll(I->c_str());
typedef void (*OptimizeFunc)(PassManager&,int);
OptimizeFunc OF = OptimizeFunc(
(intptr_t)dll.GetAddressOfSymbol(“RunOptimizations”));
if (OF == 0) {
throw std::string(“Optimization Module '”) + *I +
“’ is missing the RunOptimizations symbol”;
}
(*OF)(Passes,OptLevel);
}

The RunOptimizations-symbol is only mentioned briefly in the manual of llvm-ld:

-load module

Load an optimization module, *module*, which is expected to be a dynamic library that provides the function name `RunOptimizations`. This function will be passed the PassManager, and the optimization level (values 0-5 based on the **-On** option). This function may add passes to the PassManager that should be run. This feature allows the optimization passes of **llvm-ld** to be extended.

Now, here are my two questions (finally ;-)) :

Why is SOME_OTHER_LIB not a library from llvm-ld’s point of view? It’s supposed to be a shared library needed by opt1.so (my first optimization pass), and NOT by the source files which are processed. Should I pass it in some other way than by -lSOME_OTHER_LIB? Should/Can I pass it in the first place?
What’s the meaning of RunOptimizations? This name is only mentioned in the code snippet above and in the llvm-ld docs, but nowhere else. Both opt1.so and opt2.so do show up using opt, so it seems there is some incompatibility between opt and llvm-ld. llvm-link apparently offers no alternative.

Kind regards,

Bram Adams
GH-SEL, INTEC, Ghent University (Belgium)

Reid_Spencer · August 16, 2006, 4:59pm

Hi Bram,

Comments inline ..

Hi,

I'm using LLVM 1.8 with the GCC4-frontend on a Slackware 10.2 box (GCC
3.3.6). When issuing the following command (line 14 in a script called
manual.sh):

llvm-ld -O1 *.trans -L${SOME_LIB_DIR} -L${SOME_OTHER_LIB_DIR} -L
${LLVM_FRONT}/lib -lSOME_LIB -lSOME_OTHER_LIB -load=
${MY_LIB_DIR}/opt1.so -load=${MY_LIB_DIR}/opt2.so -debug-pass=Details
-debug -o test

... I get the following messages:

llvm-ld: warning: Supposed library 'SOME_OTHER_LIB' isn't a library.

The -l option is used to link libraries into the resulting executable.
I'm hoping "SOME_OTHER_LIB" is not the actual name and you're just
paraphrasing the actual command line. Note that native libraries are not
supported by llvm-ld at this time. It is expecting a bytecode library
that it can link into the program, hence the warning.

./manual.sh: line 14: 25492 Segmentation fault

Not sure what that is about.

When I invoke the same from within gdb, I get:

[Thread debugging using libthread_db enabled]
[New Thread -1210504992 (LWP 25520)]
llvm-ld: warning: Supposed library 'SOME_OTHER_LIB' isn't a library.

Program received signal SIGABRT, Aborted.
[Switching to Thread -1210504992 (LWP 25520)]
0xb7dbb027 in raise () from /lib/tls/libc.so.6

... with the following backtrace:

#0 0xb7dbb027 in raise () from /lib/tls/libc.so.6
#1 0xb7dbc747 in abort () from /lib/tls/libc.so.6
#2 0xb7eb6721 in uw_init_context_1 () from
${LLVM_FRONT}/lib/libgcc_s.so.1
#3 0xbf9fd140 in ?? ()
#4 0xbf9fcf20 in ?? ()
#5 0x00000064 in ?? ()
#6 0x082a02cd in llvm::Optimize (M=0x85bf330) at
${LLVM_SRC}/tools/llvm-ld/Optimize.cpp:180
#7 0x082a7161 in main (argc=17, argv=0xbf9fd454, envp=0xbf9fd49c) at
${LLVM_SRC}/tools/llvm-ld/llvm-ld.cpp:467

The exception raised apparently was not caused by the first warning
message, but resulted from the following snippet of
${LLVM_SRC}/tools/llvm-ld/Optimize.cpp (near line 180):
std::vector<std::string> plugins = LoadableModules;
for (std::vector<std::string>::iterator I = plugins.begin(), E =
plugins.end(); I != E; ++I) {
    sys::DynamicLibrary dll(I->c_str());
    typedef void (*OptimizeFunc)(PassManager&,int);
    OptimizeFunc OF = OptimizeFunc(
        (intptr_t)dll.GetAddressOfSymbol("RunOptimizations"));
    if (OF == 0) {
      throw std::string("Optimization Module '") + *I +
        "' is missing the RunOptimizations symbol";
    }
    (*OF)(Passes,OptLevel);
}

The RunOptimizations-symbol is only mentioned briefly in the manual of
llvm-ld:

-load module

        Load an optimization module, module, which is expected to be a
        dynamic library that provides the function name
        RunOptimizations. This function will be passed the
        PassManager, and the optimization level (values 0-5 based on
        the -On option). This function may add passes to the
        PassManager that should be run. This feature allows the
        optimization passes of llvm-ld to be extended.

Okay.

Now, here are my two questions (finally ;-)) :
      * Why is SOME_OTHER_LIB not a library from llvm-ld's point of
        view? It's supposed to be a shared library needed by opt1.so
        (my first optimization pass), and NOT by the source files
        which are processed. Should I pass it in some other way than
        by -lSOME_OTHER_LIB? Should/Can I pass it in the first place?

The -l option is used to specify a library to be linked with, not for
dependent symbols your loadable module requires. The loaded module must
be fully linked with all libraries it depends on. So, no, your usage
isn't correct here.

      * What's the meaning of RunOptimizations? This name is only
        mentioned in the code snippet above and in the llvm-ld docs,
        but nowhere else. Both opt1.so and opt2.so do show up using
        opt, so it seems there is some incompatibility between opt and
        llvm-ld. llvm-link apparently offers no alternative.

Yes, there is. The opt tool can run into symbol redefinition and
undefined references using its current mechanism of simply using static
constructors to define the passes to be run. llvm-ld takes a different
approach: it defines an interface function (RunOptimizations) that is
called by llvm-ld to transform the module. This gives the implementer of
the loaded module much more latitude in how the module is transformed.
Since the PassManager is passed to the RunOptimizations function as the
first argument, that function can alter the PassManager in any way it
pleases. Generally, it will add its own passes to the PassManager which
will then be run (later) by llvm-ld.

Please note that llvm-ld is an unfinished tool and you are working at
the edge of its functionality. I plan to complete this tool sometime
this year, but I have other higher priority things to work on. Also, the
LTO work may factor into a future redesign of this tool.

Hope this helps.

Reid.

Bram_Adams · August 16, 2006, 8:41pm

Hi,

llvm-ld: warning: Supposed library 'SOME_OTHER_LIB' isn't a library.

The -l option is used to link libraries into the resulting executable.
I'm hoping "SOME_OTHER_LIB" is not the actual name and you're just
paraphrasing the actual command line. Note that native libraries are not
supported by llvm-ld at this time. It is expecting a bytecode library
that it can link into the program, hence the warning.

Are there any workarounds (using other LLVM-tools) to link a native library with LLVM-bytecode?

The -l option is used to specify a library to be linked with, not for
dependent symbols your loadable module requires. The loaded module must
be fully linked with all libraries it depends on. So, no, your usage
isn't correct here.

OK.

      * What's the meaning of RunOptimizations? This name is only
        mentioned in the code snippet above and in the llvm-ld docs,
        but nowhere else. Both opt1.so and opt2.so do show up using
        opt, so it seems there is some incompatibility between opt and
        llvm-ld. llvm-link apparently offers no alternative.

Yes, there is. The opt tool can run into symbol redefinition and
undefined references using its current mechanism of simply using static
constructors to define the passes to be run. llvm-ld takes a different
approach: it defines an interface function (RunOptimizations) that is
called by llvm-ld to transform the module. This gives the implementer of
the loaded module much more latitude in how the module is transformed.
Since the PassManager is passed to the RunOptimizations function as the
first argument, that function can alter the PassManager in any way it
pleases. Generally, it will add its own passes to the PassManager which
will then be run (later) by llvm-ld.

That's interesting! So, one only needs to add a 2-arg function called RunOptimizations to the module (can't check it right now)?

Please note that llvm-ld is an unfinished tool and you are working at
the edge of its functionality.

I like living at the edge

I plan to complete this tool sometime
this year, but I have other higher priority things to work on. Also, the
LTO work may factor into a future redesign of this tool.

OK.

Hope this helps.

It sure did!

Kind regards,

Bram Adams
GH-SEL, INTEC, Ghent University (Belgium)

Reid_Spencer · August 16, 2006, 9:07pm

Hi Bram,

Are there any workarounds (using other LLVM-tools) to link a native
library with LLVM-bytecode?

Yes. The general approach is to use llvm-as, llvm-dis, opt and llvm-link
to work with the bytecode. You can also use gccld and gccas to run some
canned optimizations. llvm-link works exclusively with bytecode. Once
you have a final bytecode file you can:

llc -o file.s file.bc # generate native assembly
gcc -S file.s -o file.o
gcc file.o -lwhatever

That's interesting! So, one only needs to add a 2-arg function called
RunOptimizations to the module (can't check it right now)?

That is correct. That function and only that function will be called.
What happens in that function is up to you

Reid.

Bram_Adams · August 18, 2006, 3:49pm

Hi,

Reid Spencer wrote:

That's interesting! So, one only needs to add a 2-arg function called  
RunOptimizations to the module (can't check it right now)?


That is correct. That function and only that function will be called.
What happens in that function is up to you :)

So, I tried this the last two days, but to no avail. I first wrote the following three files to introduce the RunOptimizations-symbol (adapted from ): a) Loader.h #ifndef LOADER_H #define LOADER_H #ifdef __cplusplus #include “llvm/PassManager.h” using namespace llvm; #else typedef struct PassManager PassManager; #endif #ifdef __cplusplus extern “C” { extern void RunOptimizations(PassManager*,…); /* ANSI C prototypes / extern PassManager load_weaver_passes(PassManager*); } #endif #endif b) c-Loader.c #include “loader/Loader.h” void RunOptimizations(PassManager* mgr,…){ load_weaver_passes(mgr); } c) Loader.cpp #include “loader/Loader.h” #include “matcher/Matcher.h” #include “reifier/Reifier.h” PassManager* load_weaver_passes(PassManager* mgr){ mgr->add(createFirstPass()); mgr->add(createSecondPass()); return mgr; } In short, the C part (RunOptimizations) hands its first argument over to a global C++ method to add my passes. When using this machinery with llvm-ld (Slackware 10.2, GCC 3.3.6 and LLVM 1.8a), my passes are indeed loaded in and the PassManager attempts to run on the processed module. Then, I get the following: Program received signal SIGSEGV, Segmentation fault. 0xb7f16abf in __dynamic_cast () from /usr/lib/libstdc++.so.5 #0 0xb7ea9abf in __dynamic_cast () from /usr/lib/libstdc++.so.5 #1 0x0843f19d in llvm::PMDebug::PrintArgumentInformation (P=0x85e3b00) at /…/llvm/lib/VMCore/Pass.cpp:132 #2 0x0843f1fc in llvm::PMDebug::PrintArgumentInformation (P=0x85ca030) at /…/llvm/lib/VMCore/Pass.cpp:134 #3 0x0844e3b8 in llvm::PMDebug::PerformPassStartupStuff (P=0x85ca030) at PassManagerT.h:66 #4 0x08444146 in llvm::PassManagerTllvm::MTraits::runOnUnit (this=0x85ca048, M=0x85bf330) at PassManagerT.h:280 #5 0x0843eb26 in llvm::ModulePassManager::runOnModule (this=0x85ca030, M=@0x85bf330) at PassManagerT.h:905 #6 0x0843eebf in llvm::PassManager::run (this=0xbfe36400, M=@0x85bf330) at /…/llvm/lib/VMCore/Pass.cpp:85 #7 0x082a0361 in llvm::Optimize (M=0x85bf330) at /…/llvm/tools/llvm-ld/Optimize.cpp:189 #8 0x082a7161 in main (argc=14, argv=0xbfe365e4, envp=0xbfe36620) at /…/llvm/tools/llvm-ld/llvm-ld.cpp:467 The backtrace led me to this code snippet (lib/VMCore/Pass.cpp near line 132): void PMDebug::PrintArgumentInformation(const Pass *P) { // Print out passes in pass manager… if (const AnalysisResolver PM = dynamic_cast<const AnalysisResolver>(P)) { //<-- !!! ERROR !!! for (unsigned i = 0, e = PM->getNumContainedPasses(); i != e; ++i) PrintArgumentInformation(PM->getContainedPass(i)); } else { // Normal pass. Print argument information… // Print out arguments for registered passes that are optimizations if (const PassInfo *PI = P->getPassInfo()) if (PI->getPassType() & PassInfo::Optimization) std::cerr << " -" << PI->getPassArgument(); } } While I could verify (using GDB/DDD) that my passes were loaded perfectly in load_weaver_pass, once they got on the line marked with ERROR the pointer to their vtable suddenly was inaccessible. This made dynamic_cast crash. The hardcoded passes, loaded in before mine by llvm-ld, had no problems whatsoever. For some reason, some memory allocated in my C/C++ bridge is just not accessible anymore by llvm-ld, but I don’t know why … At long last, I decided to port opt’s pass loading mechanism to llvm-ld to see if this would solve things in my case (despite its limitations). And it did! I attached my hack (with uncommented existing loading mechanism) in case someone else encounters the same problems. Maybe some extra argument switch for llvm-ld could allow user-switchable coexistence of both loading mechanisms? Has anyone else experimented with llvm-ld yet? Did the same problem arise? Kind regards, Bram Adams GH-SEL, INTEC, Ghent University

patch.llvm-ld-1.8 (2.58 KB)

Reid_Spencer · August 20, 2006, 7:18pm

Bram,

I looked over your patch and it looks good. I applied a patch based on
yours. The llvm-ld tool now uses the PluginLoader just like the opt
tool. It will also run some cleanup passes after the loaded plugins run
to ensure cruft is removed.

See this patch for details:
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20060814/036882.html

Thanks for the patch! And, yes, you are on the bleeding edge of llvm-ld
usage. I don't think anyone else was using RunOptimizations. You're the
first to comment on it.

Reid.

Bram_Adams · August 20, 2006, 8:04pm

Hi,

I looked over your patch and it looks good. I applied a patch based on
yours. The llvm-ld tool now uses the PluginLoader just like the opt
tool. It will also run some cleanup passes after the loaded plugins run
to ensure cruft is removed.

OK, thanks. Your patch seems to work, although I also get the following unharmful warning (cf. http://llvm.cs.uiuc.edu/bugs/show_bug.cgi?id=885):

<unknown>: CommandLine Error: Argument 'verify' defined more than once!

Kind regards,

Bram Adams
GH-SEL, INTEC, Ghent University (Belgium)

Reid_Spencer · August 20, 2006, 8:26pm

Do you have a verify option in your loaded module?

Reid.

Reid_Spencer · August 20, 2006, 8:43pm

Actually, never mind .. its a result of using the PassNameParser.

Will fix shortly.

Reid.

Bram_Adams · August 20, 2006, 8:59pm

Hi,

Reid_Spencer · August 20, 2006, 11:22pm

Yeah, it wasn't your problem but mine. Please update Optimize.cpp or
apply the recent patches to it. Should be fixed now.

Reid.

Bram_Adams · August 21, 2006, 8:25am

Hi,

Reid Spencer wrote:

Yeah, it wasn't your problem but mine. Please update Optimize.cpp or
apply the recent patches to it. Should be fixed now.

Correct: no CommandLine Errors anymore.

Kind regards,

Bram Adams
GH-SEL, INTEC, Ghent University (Belgium)