[Lldb-commits] [lldb] r213158 - Add kalimba as a platform.

Subprojects LLDB
1

Hey Matt,

(Adding back the list since these are general troubleshooting tips for getting tests running).

Sorry you’re having so much trouble with this!

Ok so a few things to troubleshoot:

Regardless of the build system used, go to the build output’s lib dir. Do this command:
ldd liblldb.so

That will spit out all the shared libraries that your liblldb.so is trying to link against. The python message your seeing will happen if python can’t find the lldb module (as the message suggests and you were tracing down), or if it can find it but fails to load it (which is often the case - something cannot be found when the liblldb.so tries to load as a consequence of the lldb python module trying to load).

You should see something that looks roughly like this:

tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$ ldd liblldb.so
        linux-vdso.so.1 =>  (0x00007fffe9dfe000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6357e71000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6357b6b000)
        libedit.so.2 => /usr/lib/x86_64-linux-gnu/libedit.so.2 (0x00007f635793a000)
        libpanel.so.5 => /usr/lib/x86_64-linux-gnu/libpanel.so.5 (0x00007f6357736000)
        libncurses.so.5 => /lib/x86_64-linux-gnu/libncurses.so.5 (0x00007f6357513000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f63572e9000)
        libpython2.7.so.1.0 => /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0 (0x00007f6356d82000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6356b7a000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6356975000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6356757000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f635653e000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6356327000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6355f61000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f635d584000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f6355d5d000)
tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$

The left side shows the short name that the .so is referencing. The right side shows what the loader actually mapped it to with the current environment. If there is a problem loading liblldb.so, this will show as (I think) question marks on the right side indicating that the shared library linkage was not found. Let me know if you see something like that.

If you get past this part and you don’t see any issues, the next thing to try is:
PYTHONPATH=/path/to/your/lldb/lib/python2.7/site-packages python
Then do an ‘import lldb’, then a ‘dir (lldb)’, like so:

2

Hi Todd,

Thanks for taking the time to come up with these tips. First off my distro is Fedora Core 20 64-bit. So to further my investigation I blew away all the sources, and did svn co llvm, (lldb clang into llvm/tools), made the sibling build dir, then did:

~/src/p4play/build
$ ../llvm/configure --enable-targets=x86_64 --enable-shared --enable-cxx11 --prefix=`pwd`/../install && make -j 8

After the successful build I do ldd. Basically lld finds all dependencies for liblldb.so...

~/src/p4play/build/Debug+Asserts/lib
$ ldd liblldb.so
     linux-vdso.so.1 => (0x00007fffa61fe000)
     libLLVM-3.5svn.so => /home/mg11/src/p4play/build/Debug+Asserts/lib/./libLLVM-3.5svn.so (0x00007f2e53405000)
     libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f2e531cc000)
     libdl.so.2 => /lib64/libdl.so.2 (0x00007f2e52fc8000)
     libutil.so.1 => /lib64/libutil.so.1 (0x00007f2e52dc5000)
     libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f2e529fe000)
     librt.so.1 => /lib64/librt.so.1 (0x00007f2e527f6000)
     libedit.so.0 => /lib64/libedit.so.0 (0x00007f2e525ba000)
     libncurses.so.5 => /lib64/libncurses.so.5 (0x00007f2e52392000)
     libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f2e52168000)
     libpanel.so.5 => /lib64/libpanel.so.5 (0x00007f2e51f64000)
     libz.so.1 => /lib64/libz.so.1 (0x00007f2e51d4d000)
     libm.so.6 => /lib64/libm.so.6 (0x00007f2e51a46000)
     libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2e5173e000)
     libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2e51527000)
     libc.so.6 => /lib64/libc.so.6 (0x00007f2e51168000)
     /lib64/ld-linux-x86-64.so.2 (0x0000003675a00000)

However, regarding python loading the lldb bindings I see my build/Debug+Asserts has a lib and a lib64 folder.

~/src/p4play/build/Debug+Asserts
$ ls
bin lib lib64

If I try to import lldb when I have "lib" as my PYTHONPATH i.e.

~/src/main/build/Debug+Asserts/lib
$ PYTHONPATH=`pwd`/python2.7/site-packages/ python
>>> import lldb
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ImportError: No module named lldb
>>>

This is not surprising, since ~/src/main/build/Debug+Asserts/lib/python2.7/site-packages/ only contains readline.so.
So I cd to lib64. This looks more promising since there is a lldb folder below site-packages. So now I do "import lldb" with "lib64" instead:

~/src/main/build/Debug+Asserts/lib64
$ PYTHONPATH=`pwd`/lib64/python2.7/site-packages/ python
Python 2.7.5 (default, Feb 19 2014, 13:47:28)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lldb
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py", line 52, in <module>
     _lldb = swig_import_helper()
   File "/home/mg11/src/main/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py", line 44, in swig_import_helper
     import _lldb
ImportError: No module named _lldb
>>>

So cd to ~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb, and list:

~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb
$ ls -l
total 1196
-rw-rw-r-- 1 mg11 mg11 3905 Jul 21 08:25 embedded_interpreter.py
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 08:25 formatters
-rw-rw-r-- 1 mg11 mg11 495675 Jul 21 08:25 __init__.py
-rw-rw-r-- 1 mg11 mg11 708200 Jul 21 09:09 __init__.pyc
lrwxrwxrwx 1 mg11 mg11 19 Jul 21 08:25 _lldb.so -> ../../../liblldb.so !BROKEN SYMLINK!
drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 runtime
drwxrwxr-x 2 mg11 mg11 4096 Jul 21 08:25 utils

_lldb.so is broken above as the link above assumes liblldb.so is just 3 dirs back. However, it isn't - because it's not in lib64, *it's in lib*. So it looks like things are getting a bit mashed up due to 32-bit/64-bit directory issues. If I copy the .so files from lib to lib64, then it fixes my broken sym-link issue. And I progress a little further.

~/src/p4play/build/Debug+Asserts/lib64
$ PYTHONPATH=`pwd`/python2.7/site-packages python
Python 2.7.5 (default, Feb 19 2014, 13:47:28)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lldb
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py", line 52, in <module>
     _lldb = swig_import_helper()
   File "/home/mg11/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages/lldb/__init__.py", line 48, in swig_import_helper
     _mod = imp.load_module('_lldb', fp, pathname, description)
ImportError: libLLVM-3.5svn.so: cannot open shared object file: No such file or directory
>>>

So now I push the location of these .sos into my environment with LD_LIBRARY_PATH. Finally lldb imports!

~/src/p4play/build/Debug+Asserts/lib64
$ LD_LIBRARY_PATH=. PYTHONPATH=`pwd`/python2.7/site-packages python
Python 2.7.5 (default, Feb 19 2014, 13:47:28)
[GCC 4.8.2 20131212 (Red Hat 4.8.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lldb
>>> dir(lldb)
['INT32_MAX', 'LLDB_ARCH_DEFAULT',
... lots more ...
>>>

So, I think the crux of the problem I'm seeing here is that on a 64-bit linux build (using autoconf/make) we get the binaries distributed around the lib and lib64 sub-folders without adequate coordination with the test tools. I'll retry now with cmake/ninja and see if my experience differs.

Matt

Todd Fiala wrote:

3

Todd,

I've since discovered that the cmake/ninja build system also results in a lib and lib64 directory, with the liblldb.so and other .sos in the lib folder but the python2.7/site-packages/lldb under lib64. (In fact with cmake build I see no python2.7 folder at all under lib, it's all under lib64).

So Todd, focusing back on autoconf/make builds, after doing the ./configure && make in the sibling build directory, do you see for your linux 64-bit builds the python2.7/site-packages/lldb under the lib64 sub-folder and the lldb/llvm .sos under the lib64 sub-folder?

How do you get this to work:

tfiala@tfiala2:/mnt/ssd/work/macosx.sync/mbp-svn/build-debug/lib$ PYTHONPATH=`pwd`/python2.7/site-packages python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lldb
>>> dir(lldb)

Do you run another script or make target after doing the basic llvm/lldb make? That is to move python2.7/site-packages/lldb underneath lib. I think that's what I'd have to do on my system, since after the basic make my lib and lib64 directories have the following contents:

~/src/p4play/build/Debug+Asserts/lib/python2.7/site-packages

$ ls -al
total 32
drwxrwxr-x 2 mg11 mg11 4096 Jul 21 14:40 .
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:40 ..
-rwxr-xr-x 1 mg11 mg11 21795 Jul 21 14:40 readline.so

$ cd ../../../lib64/python2.7/site-packages/

~/src/p4play/build/Debug+Asserts/lib64/python2.7/site-packages

$ ls -al
total 12
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 .
drwxrwxr-x 3 mg11 mg11 4096 Jul 21 14:42 ..
drwxrwxr-x 5 mg11 mg11 4096 Jul 21 14:42 lldb

Any ideas?
Matt

Matthew Gardiner wrote:

4

Hey Matthew,

Great investigating! Glad you were able to at least find the core of the issue, which does look like at least on Fedora Core 20 x86_64 we are doing something funky with configure/make. I’m not sure why we’re choosing lib64 there and lib on Ubuntu 14.04 x86_64, might have to do with how the python is configured. I think that directory may be coming from a helper library in python that determines the correct location for build output and has to do with some Python finalizer scripts during the build process. I won’t be totally surprised if this might just be broken on Fedora Core x86_64 regardless of cmake or configure-based builds.

I’ll do a configure/make build on Ubuntu 14.04 x86_64 later today to see if I’m getting the same results on configure/make - there has been some changes in python handling more recently, generally to support Windows builds, but we may have unintentionally introduced that via those changes. At least that way we can see if its consistent with configure/make or if it might just be specific to Fedora Core.

Thanks for identifying that!

-Todd

5

My guess is that Fedora’s python build reports a different location for where to install the libraries to. You can debug this by changing the CMake to run sh -x when it calls the finalize swig wrapper classes shell scripts. It might also be worth seeing what happens if you define the CMake variable to use the new python-based SWIG API generation. AFAIK nobody’s ever tested this on non-Windows, but might be worth checking out.

6

I'll do a configure/make build on Ubuntu 14.04 x86_64 later today to see
if I'm getting the same results on configure/make - there has been some
changes in python handling more recently, generally to support Windows
builds, but we may have unintentionally introduced that via those changes.
At least that way we can see if its consistent with configure/make or if
it might just be specific to Fedora Core.

Okay, I tried a configure/make on top of tree on Ubuntu 14.04 x86_64. I
did not get the lib64 dir, so this seems to be a diff in the flow between
the Ubuntu and FC build.

Incidentally, I am having trouble running tests with configure/make. Might
just be my setup or configure invocation, but I'll get around to tracking
that down here at some point. For now, I'd definitely recommend going the
cmake/ninja route for testing.

7

"You can debug this by changing the CMake to run sh -x when it calls the finalize swig wrapper classes shell scripts."

Would that be editing a CMakeLists.txt file? Which one - i.e. the checked out path?

"It might also be worth seeing what happens if you define the CMake variable to use the new python-based SWIG API generation."

Whats that CMake variable called? I can't see anything mentioning swig in http://llvm.org/docs/CMake.html

thanks
Matt

Zachary Turner wrote:

8

Todd Fiala wrote:

    I'll do a configure/make build on Ubuntu 14.04 x86_64 later today
    to see if I'm getting the same results on configure/make - there
    has been some changes in python handling more recently, generally
    to support Windows builds, but we may have unintentionally
    introduced that via those changes. At least that way we can see
    if its consistent with configure/make or if it might just be
    specific to Fedora Core.

Okay, I tried a configure/make on top of tree on Ubuntu 14.04 x86_64. I did not get the lib64 dir, so this seems to be a diff in the flow between the Ubuntu and FC build.

Ok, thanks for confirming the difference between your ubuntu and my FC experience.

By the way I've just opened llvm/tools/lldb/CMakeLists.txt

if ( CMAKE_SYSTEM_NAME MATCHES "Windows" )
   set(LLDB_DEFAULT_DISABLE_PYTHON 1)
   set(LLDB_DEFAULT_DISABLE_CURSES 1)
   if (LLDB_DISABLE_PYTHON)
     set(LLDB_DEFAULT_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION 0)
   else()
     set(LLDB_DEFAULT_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION 1)
   endif()
else()
   set(LLDB_DEFAULT_DISABLE_PYTHON 0)
   set(LLDB_DEFAULT_DISABLE_CURSES 0)
   set(LLDB_DEFAULT_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION 0)
endif()

I'm guessing

set(LLDB_DEFAULT_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION 0) means:

"In non-windows we use shell scripts not python to run swig".

Anyway, I guess this is the CMake variable Zachary meant.

Incidentally, I am having trouble running tests with configure/make. Might just be my setup or configure invocation, but I'll get around to tracking that down here at some point. For now, I'd definitely recommend going the cmake/ninja route for testing.

Yeah, agreed, at least till we get this figured out.

9

The one with “default” in the name just controls the value if you don’t specify it on the command line. Below that though you see where it sets the real variable, which if I remember correctly is called LLDB_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION. You can set this from the command line by running cmake -DLLDB_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION=1

Regarding your other question, in scripts\CMakeLists.txt you’ll see these two lines:

COMMAND env PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/build-swig-wrapper-classes.sh ${LLDB_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR} -m
COMMAND env PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/finish-swig-wrapper-classes.sh ${LLDB_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR} -m

Change them to:

COMMAND env PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE} sh -x ${CMAKE_CURRENT_SOURCE_DIR}/build-swig-wrapper-classes.sh ${LLDB_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR} -m
COMMAND env PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE} sh -x ${CMAKE_CURRENT_SOURCE_DIR}/finish-swig-wrapper-classes.sh ${LLDB_SOURCE_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR} -m

10

Thanks Zachary. I think I've figured out now, though... I'm just about to post an answer in another mail :slight_smile:

Zachary Turner wrote:

11

More investigating reveals that it is finish-swig-Python-LLDB.sh which is lldb's responsible script. (Thanks for the -x reminder, Zachary!). The line in the script is

framework_python_dir=`${PYTHON} -c "from distutils.sysconfig import get_python_lib; print get_python_lib(True, False, \"${PYTHON_INSTALL_DIR}\");"`/lldb

When I run the raw invocation at my shell:

$ /usr/bin/python -c 'from distutils.sysconfig import get_python_lib; print get_python_lib(True, False, "/home/mg11/src/p4play/build");'
/home/mg11/src/p4play/build/lib64/python2.7/site-packages

So, we might suspect that FC20s python install is wrong.

However, I downloaded python-2.7.7 and built from source, with a pretty much a raw config (only configure tweaks being enable shared and threads), and I get a similar output:

$ /usr/local/Python-2.7.7/bin/python -c 'from distutils.sysconfig import get_python_lib; print get_python_lib(True, False, "/home/mg11/src/p4play/build");'
/home/mg11/src/p4play/build/lib64/python2.7/site-packages

I may well be sticking my neck out a bit here... but I'm wondering if lib64 is actually the __right__ answer here (given my 64-bit host). Is python trying to support 32-bit binaries in lib and 64-bit ones in lib64 or something?

(For *NIX 64-bit builds should our llvm/lldb binaries go in lib64? not lib)

I'm curious as to what people think here... so we can engineer as nice a fix as possible.

Matt

Todd Fiala wrote:

12

Guys,

This little tweak fixes things for me. I.e. both the llvm/lldb .sos and the python2.7 contents get written under build/lib. (Not partially under lib64.)
I'm not sure whether it's the right fix, though. Or whether it has the potential to break other systems.

Matt

Index: scripts/Python/finish-swig-Python-LLDB.sh

13

Can you ignore my assumption about the vanilla python behaviour regarding distutils.sysconfig? I need to investigate it further. I *now* that my local /usr/local/Python-2.7.7 install somehow reaches in my default FC20 python to get the "lib64" string.

thanks
Matt

Matthew Gardiner wrote:

However, I downloaded python-2.7.7 and built from source, with a pretty much a raw config (only configure tweaks being enable shared and threads), and I get a similar output:

$ /usr/local/Python-2.7.7/bin/python -c 'from distutils.sysconfig import get_python_lib; print get_python_lib(True, False, "/home/mg11/src/p4play/build");'
/home/mg11/src/p4play/build/lib64/python2.7/site-packages

I may well be sticking my neck out a bit here... but I'm wondering if lib64 is actually the __right__ answer here (given my 64-bit host). Is python trying to support 32-bit binaries in lib and 64-bit ones in lib64 or something?

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.