LLD dynamic compilation

Folks,

I'm having a look at LLD and I need some guidance...

I know it's not production ready for x86 and ARM (the idea is to make
it so). My steps:

I've added it to tools/lld and ran CMake again (on x86_64) on a
standard release build (static linking). It works, builds but I see
one unit test error:

Note: Google Test filter = InputGraphTest.Observer
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from InputGraphTest
[ RUN ] InputGraphTest.Observer
0 DriverTests 0x0000000000498f42
1 DriverTests 0x0000000000498c24
2 libpthread.so.0 0x00007f0ecbf7f4b0
3 DriverTests 0x0000000000462f6a
4 DriverTests 0x00000000005bc3a4
5 DriverTests 0x000000000046bb6f
6 DriverTests 0x0000000000478eea
7 DriverTests 0x0000000000479f58
8 DriverTests 0x000000000047a025
9 DriverTests 0x000000000047e5d0
10 DriverTests 0x000000000047e882
11 DriverTests 0x0000000000411656
12 libc.so.6 0x00007f0ecb1a3000 __libc_start_main + 240
13 DriverTests 0x00000000004169a7

When trying to investigate, I then ran CMake on my debug build (that
has dynamic linking to avoid re-linking of huge libraries during
development cycle), and I got this:

CMake Error: The inter-target dependency graph contains the following
strongly connected component (cycle):
  "lldPasses" of type SHARED_LIBRARY
    depends on "lldNative" (weak)
    depends on "lldYAML" (weak)
  "lldNative" of type SHARED_LIBRARY
    depends on "lldYAML" (weak)
    depends on "lldPasses" (weak)
  "lldYAML" of type SHARED_LIBRARY
    depends on "lldNative" (weak)
    depends on "lldPasses" (weak)
At least one of these targets is not a STATIC_LIBRARY. Cyclic
dependencies are allowed only among static libraries.

The error is clear, and the only way (I know) to make it compile
dynamically is to remove the cyclic dependency.

Question is: Why the cyclic dependency? Is that really necessary, or
is that no one ever bothered?

cheers,
--renato

Hi Renato,

I think you are hitting a bug, the Observer pattern was added a few weeks back, and may be there is some sort of uninitialized variable ?

On the cyclic dependency there was a recent conversation that was trying to fix this, but I am not sure where it is.

Thanks

Shankar Easwaran

I think you are hitting a bug, the Observer pattern was added a few weeks
back, and may be there is some sort of uninitialized variable ?

This is my back-trace at "-O2 -g" (since -O1 pass):

operator() (file=<optimized out>, __closure=0x7fffffffde40) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:197
197 _graph->registerObserver([&](File *file) {
files.push_back(file->path()); });
(gdb) bt
#0 operator() (file=<optimized out>, __closure=0x7fffffffde40) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:197
#1 llvm::function_ref<void(lld::File*)>::callback_fn<InputGraphTest_Observer_Test::TestBody()::<lambda(lld::File*)>

(intptr_t, lld::File *) (callable=140737488346688,

    param1=<optimized out>) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:117
#2 0x00000000005b3ec4 in operator() (param1=<optimized out>,
this=0xa1e2a0) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:126
#3 callback_fn<llvm::function_ref<void(lld::File*)> >
(param1=<optimized out>, callable=10609312) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:117
#4 operator() (param1=<optimized out>, this=<synthetic pointer>) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:126
#5 lld::InputGraph::getNextFile (this=<optimized out>) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/lib/Core/InputGraph.cpp:27
#6 0x0000000000462a87 in (anonymous
namespace)::InputGraphTest::getNext (this=0xa1e6b0)
    at /home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:64
#7 0x000000000046c749 in InputGraphTest_Observer_Test::TestBody
(this=0xa1e6b0) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:201
#8 0x0000000000479bda in testing::Test::Run (this=0xa1e6b0) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2162
#9 0x000000000047abd0 in testing::TestInfo::Run (this=0xa1d6f0) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2309
#10 0x000000000047adc5 in testing::TestCase::Run (this=0xa1cfe0) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2416
#11 0x000000000047fd40 in testing::internal::UnitTestImpl::RunAllTests
(this=0xa0f140) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:4205
#12 0x000000000047fff2 in
HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,

(location=<synthetic pointer>,

    method=(bool
(testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl *
const)) 0x47faf0 <testing::internal::UnitTestImpl::RunAllTests()>,
object=0xa0f140)
    at /home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2145
#13 testing::UnitTest::Run (this=<optimized out>) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:3842
#14 0x0000000000410796 in main (argc=1, argv=0x7fffffffe0d8) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/UnitTestMain/TestMain.cpp:48

On the cyclic dependency there was a recent conversation that was trying to
fix this, but I am not sure where it is.

Is this on llvmdev? Or is there a special lld list?

cheers,
--renato

I think you are hitting a bug, the Observer pattern was added a few weeks
back, and may be there is some sort of uninitialized variable ?

This is my back-trace at "-O2 -g" (since -O1 pass):

operator() (file=<optimized out>, __closure=0x7fffffffde40) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:197
197 _graph->registerObserver([&](File *file) {
files.push_back(file->path()); });
(gdb) bt
#0 operator() (file=<optimized out>, __closure=0x7fffffffde40) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:197
#1 llvm::function_ref<void(lld::File*)>::callback_fn<InputGraphTest_Observer_Test::TestBody()::<lambda(lld::File*)>

(intptr_t, lld::File *) (callable=140737488346688,

     param1=<optimized out>) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:117
#2 0x00000000005b3ec4 in operator() (param1=<optimized out>,
this=0xa1e2a0) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:126
#3 callback_fn<llvm::function_ref<void(lld::File*)> >
(param1=<optimized out>, callable=10609312) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:117
#4 operator() (param1=<optimized out>, this=<synthetic pointer>) at
/home/rengolin/devel/llvm/src/llvm/include/llvm/ADT/STLExtras.h:126
#5 lld::InputGraph::getNextFile (this=<optimized out>) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/lib/Core/InputGraph.cpp:27
#6 0x0000000000462a87 in (anonymous
namespace)::InputGraphTest::getNext (this=0xa1e6b0)
     at /home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:64
#7 0x000000000046c749 in InputGraphTest_Observer_Test::TestBody
(this=0xa1e6b0) at
/home/rengolin/devel/llvm/src/llvm/tools/lld/unittests/DriverTests/InputGraphTest.cpp:201
#8 0x0000000000479bda in testing::Test::Run (this=0xa1e6b0) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2162
#9 0x000000000047abd0 in testing::TestInfo::Run (this=0xa1d6f0) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2309
#10 0x000000000047adc5 in testing::TestCase::Run (this=0xa1cfe0) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2416
#11 0x000000000047fd40 in testing::internal::UnitTestImpl::RunAllTests
(this=0xa0f140) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:4205
#12 0x000000000047fff2 in
HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,
> (location=<synthetic pointer>,
     method=(bool
(testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl *
const)) 0x47faf0 <testing::internal::UnitTestImpl::RunAllTests()>,
object=0xa0f140)
     at /home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:2145
#13 testing::UnitTest::Run (this=<optimized out>) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/googletest/src/gtest.cc:3842
#14 0x0000000000410796 in main (argc=1, argv=0x7fffffffe0d8) at
/home/rengolin/devel/llvm/src/llvm/utils/unittest/UnitTestMain/TestMain.cpp:48

Thanks Renato.

On the cyclic dependency there was a recent conversation that was trying to
fix this, but I am not sure where it is.

Is this on llvmdev? Or is there a special lld list?

http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-June/073407.html

Shankar Easwaran

Hi Shankar,

It seems that the "solution" proposed on that email was done, but it
doesn't fix the cyclic dependency in CMake.

I'm copying the parties involved to make sure it's not something I am
doing wrong.

cheers,
--renato

I tried to debug this usecase f as I am unable to reproduce it on my ubuntu machine, with the stack trace that you have mentioned above, I dont see how this could result here.

Could you try running this with sanitize options and check if there is something that your setup is figuring out something wrong ?

Thanks

Shankar Easwaran