LLDB test executable architectures

Currently dotest.py contains the following logic to determine what architectures to compile the test executables as:

if args.archs:

architectures were specified on the command line, just use them

else:

if (platform_system == ‘Darwin’ or (platform_system == ‘Linux’ and compilers == [‘clang’])) and platform_machine == ‘x86_64’:
archs = [‘x86_64’, ‘i386’]
else:
archs = [platform_machine]

Does anyone actually need this kind of complicated logic? It’s kind of magical and hand-wavy. There’s no indication of why it makes sense that Darwin+x64 system would default to running both x64 and x86 tests, or why linux gcc x64 would run only x64 tests but not x86 tests, even though linux clang x64 would run both sets of tests.

I’d like to simplify it if possible (partly because this logic is actually broken on Windows, so I need to revisit it anyway). Is there any reason we can’t just keep it as simple as “If it’s on the command line, use it, otherwise default to running only the tests corresponding to the system platform?”

If the logic is broken, please fix, but don't remove or simplify it just because it is complex. Make sure that if a platform (like darwin) supports both x86_64 and i386 binaries, that the tests run for both so we cover all bases and know if something fails for 32 or 64 bit. Sounds like on Windows you only want to run x86_64 for 64 bit machines or i386 for 32 bit machine right?

Just make sure Darwin runs both with what ever fix you make.

Well I guess it would be helpful to know how you run the tests. Do you run dotest.py from the command line? Or do you have a tool that drives the script? Because if it’s the latter, then the tool can just pass in whatever architectures it wants. I have a patch to the CMake build right now that makes the CMake build always pass in the target architectures. So that will remove the need for this logic for anyone running tests via CMake. But I’m not sure what you do on Mac.

I guess what I’m saying is that complicated logic is fine if it’s useful. I just don’t know if it’s useful (maybe it is, but I don’t know what the workflow is like on Mac). If you guys are already running all the tests via a tool that passes in --arch on the command line, or if you’re willing to change whatever tool you do use (the Xcode project?) to pass in --arch, then the logic here probably isn’t that useful.

Just recently I figured out how to successfully run dotest.py on a 64-bit linux. I ran it from the command line as follows:

$ my-lldb-source-directory/test >
LD_LIBRARY_PATH=/my-build-directory/lib/
PYTHONPATH=/my-build-directory/lib/python2.7/site-packages/
python dotest.py --executable=/my-build-directorybin/lldb -v --compiler=gcc -q .

The above incarnation took some time to figure out!

I guess what I'm saying is if dotest.py is changed in such a way that it needs to be run using another tool, and/or if the -arch setting must be passed in, then this should be documented *within* dotest.py itself.

Zachary Turner wrote:

Well I guess it would be helpful to know how you run the tests. Do you run dotest.py from the command line? Or do you have a tool that drives the script? Because if it's the latter, then the tool can just pass in whatever architectures it wants. I have a patch to the CMake build right now that makes the CMake build always pass in the target architectures. So that will remove the need for this logic for anyone running tests via CMake. But I'm not sure what you do on Mac.

I guess what I'm saying is that complicated logic is fine if it's useful. I just don't know if it's useful (maybe it is, but I don't know what the workflow is like on Mac). If you guys are already running all the tests via a tool that passes in --arch on the command line, or if you're willing to change whatever tool you do use (the Xcode project?) to pass in --arch, then the logic here probably isn't that useful.

    If the logic is broken, please fix, but don't remove or simplify
    it just because it is complex. Make sure that if a platform (like
    darwin) supports both x86_64 and i386 binaries, that the tests run
    for both so we cover all bases and know if something fails for 32
    or 64 bit. Sounds like on Windows you only want to run x86_64 for
    64 bit machines or i386 for 32 bit machine right?

    Just make sure Darwin runs both with what ever fix you make.

    >
    > Currently dotest.py contains the following logic to determine
    what architectures to compile the test executables as:
    >
    > if args.archs:
    > # architectures were specified on the command line, just
    use them
    > else:
    > if (platform_system == 'Darwin' or (platform_system ==
    'Linux' and compilers == ['clang'])) and platform_machine == 'x86_64':
    > archs = ['x86_64', 'i386']
    > else:
    > archs = [platform_machine]
    >
    > Does anyone actually need this kind of complicated logic? It's
    kind of magical and hand-wavy. There's no indication of why it
    makes sense that Darwin+x64 system would default to running both
    x64 and x86 tests, or why linux gcc x64 would run only x64 tests
    but not x86 tests, even though linux clang x64 would run both sets
    of tests.
    >
    > I'd like to simplify it if possible (partly because this logic
    is actually broken on Windows, so I need to revisit it anyway).
     Is there any reason we can't just keep it as simple as "If it's
    on the command line, use it, otherwise default to running only the
    tests corresponding to the system platform?"
    > _______________________________________________
    > lldb-dev mailing list
    > lldb-dev@cs.uiuc.edu <mailto:lldb-dev@cs.uiuc.edu>
    > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev

To report this email as spam click here <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==>.

_______________________________________________
lldb-dev mailing list
lldb-dev@cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.

So just to be clear, assuming I made this change, I’m not saying that dotest would require you to pass in --arch, I’m just saying it would pick a more straightforward default. In particular, the default of your host system. You can see the original logic for choosing the default in the OP, but it was something along the lines of:

x64 Mac → x86 + x64 tests
x86 Mac → x86 tests
x64 Linux Clang → x86 + x64 tests
x64 Linux Non-Clang (GCC?) x64 tests
x86 Linux Clang → x86 tests
x86 Linux Non-Clang → x86 tests

So you’d have the following net changes:

x64 Mac → Don’t run x86 tests anymore
x64 Linux Clang → Don’t run x86 tests anymore

Nothing else would change. And again, these would just be for the cases where you ran dotest.py without a --arch arg. You could still override it.

If it’s truly useful then I don’t mind leaving it, but I’d at least like to understand why it’s so asymmetric. Like what’s the issue with Linux GCC x64? Why can’t it run x86 tests?

Zachary Turner wrote:

So just to be clear, assuming I made this change, I'm not saying that dotest would require you to pass in --arch, I'm just saying it would pick a more straightforward default. In particular, the default of your host system. You can see the original logic for choosing the default in the OP, but it was something along the lines of:

x64 Mac -> x86 + x64 tests
x86 Mac -> x86 tests
x64 Linux Clang -> x86 + x64 tests
x64 Linux Non-Clang (GCC?) x64 tests
x86 Linux Clang -> x86 tests
x86 Linux Non-Clang -> x86 tests

So you'd have the following net changes:

x64 Mac -> Don't run x86 tests anymore
x64 Linux Clang -> Don't run x86 tests anymore

Nothing else would change. And again, these would just be for the cases where you ran dotest.py without a --arch arg. You could still override it.

Ok, fair enough - thanks for putting my mind at rest. Clearly the process should not be more difficult than it already is.

If it's truly useful then I don't mind leaving it, but I'd at least like to understand why it's so asymmetric. Like what's the issue with Linux GCC x64? Why can't it run x86 tests?

I see what you mean. I'm not sure why CLANG builds test both architectures. Perhaps the CLANG environment has features which make it easier to run 32 stuff on 64 OSes? (I know very little about clang/llvm). I'm making this claim because I know that on my Fedora 20 64-bit box, with stuff built on GCC, I had to install additional (i.e. 32 bit support libraries) to get C and C++ binaries to run under the 64-bit linux. So perhaps there is a reason?

Yes I just run:

cd lldb/test
./dotest.py

And by default I would hope that "detest.py" would be able to be run (as well as dosep.py) on its own with no arguments and it should do the right thing for the currently built lldb.

We should fix the detest.py to possibly look at the configure logs or do something so it can figure out all the right stuff if that is possible by looking at the lldb source directory. If this isn't possible, we should provide a nice message on each platform with the common things we need to supply. Matthew, feel free to add code to dotest.py that will make this clearer for future test runners.

Greg

That sounds fine to me.

We really do have to come up with a reason to have a “detest.py” script!

Jim

Greg Clayton wrote:

We should fix the detest.py to possibly look at the configure logs or do something so it can figure out all the right stuff if that is possible by looking at the lldb source directory. If this isn't possible, we should provide a nice message on each platform with the common things we need to supply. Matthew, feel free to add code to dotest.py that will make this clearer for future test runners.

Ok.. I'll look into this, at some stage. Given the different build systems and the different OSes, the "nice message" option. But I'll want I can do.

Matt

Just recently I figured out how to successfully run dotest.py on a 64-bit linux. I ran it from the command line as follows:

$ my-lldb-source-directory/test >
LD_LIBRARY_PATH=/my-build-directory/lib/
PYTHONPATH=/my-build-directory/lib/python2.7/site-packages/
python dotest.py --executable=/my-build-directorybin/lldb -v --compiler=gcc -q .

The above incarnation took some time to figure out!

<..snipped>

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.