lldb tests

Hi all,

I'm trying to run the test suite to check some patches that I've been working on.
Are the expected failures updated? I have the latest XCode (4.3.2), as well as the latest command line tools.

This test always hangs, even with a clean checkout (it seemed to me that it was due to an infinite loop in the type summary):

cd lldb/test
make -C functionalities/data-formatter/data-formatter-stl/libcxx/map
./dotest.py -A x86_64 -C clang -v -t -f LibcxxMapDataFormatterTestCase.data_formatter_commands

Regards,

  Filipe

Failure-TestDataFormatterLibccMap.LibcxxMapDataFormatterTestCase.data_formatter_commands.log (6.39 KB)

Hi.
Not sure what’s wrong:

Enricos-MacBook-Pro:test egranata$ make -C functionalities/data-formatter/data-formatter-stl/libcxx/map
clang++ -gdwarf-2 -O0 -arch x86_64 -stdlib=libc++ -O0 -c -o main.o main.cpp
clang++ -gdwarf-2 -O0 -arch x86_64 -stdlib=libc++ main.o -o “a.out”
dsymutil -o “a.out.dSYM” “a.out”
Enricos-MacBook-Pro:test egranata$ ./dotest.py -A x86_64 -C clang -v -t -f LibcxxMapDataFormatterTestCase.data_formatter_commands
LLDB build dir: /Volumes/work/egranata/lldb/build/Debug
LLDB-135
Path: /Volumes/work/egranata/lldb
URL: https://enrico@llvm.org/svn/llvm-project/lldb/trunk
Repository Root: https://enrico@llvm.org/svn/llvm-project
Repository UUID: 91177308-0d34-0410-b5e6-96231b3b80d8
Revision: 153542
Node Kind: directory
Schedule: normal
Last Changed Author: enrico
Last Changed Rev: 153541
Last Changed Date: 2012-03-27 14:49:20 -0700 (Tue, 27 Mar 2012)

Session logs for test failures/errors/unexpected successes will go into directory ‘2012-03-28-08_07_33’
Command invoked: python ./dotest.py -A x86_64 -C clang -v -t -f LibcxxMapDataFormatterTestCase.data_formatter_commands
compilers=[‘clang’]

Configuration: arch=x86_64 compiler=clang

Hi Enrico,

That's weird. Let's try to find out what went wrong. Previously I was using an up-to-date git checkout, but now I checked out the trunk directly from subversion so we can be sure we're on the same revision.

I have:
Mac OS X 10.7.3
swig 2.0.4
Python 2.7.1
libedit.3.0.dylib
Apple clang version 3.1 (tags/Apple/clang-318.0.58)

I build lldb with xcodebuild -configuration Debug
For the x86_64 arch:
file build/Debug/lldb
build/Debug/lldb: Mach-O 64-bit executable x86_64

If I try a BuildAndIntegration build (without the -configuration flag), I get this error when running lldb:
[trunk:153576//test]% ../build/BuildAndIntegration/lldb
dyld: Library not loaded: @rpath/LLDB.framework/LLDB
Referenced from: /Users/filcab/Desktop/lldb/test/../build/BuildAndIntegration/lldb
Reason: image not found

Since I usually use the Debug configuration, I have no idea what's going on. LLDB.framework is in its place (next do lldb) and has the LLDB symlink and file in place, too.

Any clues?

Regards,

Filipe

  Filipe

Filipe,

You should be using the "Release" configuration. The "BuildAndIntegration" configuration is for when you intend to build and install the binary into /usr/bin/lldb with the framework in /System/Library/Frameworks/LLDB.framework.

The @rpath is a trick that lets the dynamic loader find shared libraries that are relative to each other in directories. When "lldb" is in "/usr/bin" the relative rpath for LLDB.framework is contained in a load command:

otool -lv /usr/bin/lldb
...
Load command 17
          cmd LC_RPATH
      cmdsize 64
         path @loader_path/../../System/Library/PrivateFrameworks (offset 12)
Load command 18

So, the "Debug" and "Release" builds set the rpath so that lldb is expected to be in the build folder. The "BuildAndIntegration" will set the rpath and expects you to install your binaries and this is typically only used for Apple builds that produce tools for release.

Greg Clayton

Hi,

a good first step is for you to run the test in verbose mode and attach the output.
You can also try to manually repeat the test case behavior and seeing what you get.

Moving from there should not be too complicated.

As for tool versions, I am using a previous version of swig because of licensing issues. I also have a more recent clang based off LLVM 3.1 svn. I am not sure why that would be the case but given that the test case at fault here is related to libc++ I would guess that the build of libc++ has something to do with it. However, we are on the same OSX version.

Thanks.

Enrico Granata
:email: egranata@.com
✆ (four oh eight) 862-7683

Hi Enrico,

That solves the @rpath mystery, thanks.
I will try with the release version and see if I get the same error.

Thanks,

  Filipe

Hi,

Attahced is the output of the test ran in verbose mode.
There's a very weird 'foo' that is shown after I enabled logging in the test (I enables verbose formatter logging right before the instruction that fails).

That print comes from stdmap_SynthProvider.get_child_at_index(), on line 546 of the Python/libcxx.py file. It seems like some kind of error, maybe you know what it's about (Something unfinished in that file?).

  Filipe

verbose-test.out (17.8 KB)

Failure-TestDataFormatterLibccMap.LibcxxMapDataFormatterTestCase.data_formatter_commands.log (6.45 KB)

Hi,
The ‘foo’ is indeed a left-over debugging printout. I usually remove them before committing the files, but every once in a while one remains. I will replace it with a Logger call later today.
If you actually read the code for libcxx.py:

	def get_child_at_index(self,index):
		logger = Logger.Logger()
		if index < 0:
			return None
		if index >= self.num_children():
			return None;
		if self.garbage:
			return None
		try:
			iterator = stdmap_iterator(self.root_node,max_count=self.num_children())
			# the debug info for libc++ std::map is such that __begin_node_ has a very nice and useful type
			# out of which we can grab the information we need - every other node has a less informative
			# type which omits all value information and only contains housekeeping information for the RB tree
			# hence, we need to know if we are at a node != 0, so that we can still get at the data
			need_to_skip = (index > 0)
			current = iterator.advance(index)
			if current == None:
				self.garbage = True
				return None
			if self.get_data_type():
				if not(need_to_skip):
					current = current.Dereference()
					obj = current.GetChildMemberWithName('__value_')
					obj_data = obj.GetData()
					self.get_value_offset(current) # make sure we have a valid offset for the next items
					# we do not return __value_ because then we would end up with a child named
					# __value_ instead of [0]
					return self.valobj.CreateValueFromData('[' + str(index) + ']',obj_data,self.data_type)
				else:
					# FIXME we need to have accessed item 0 before accessing any other item!
					if self.skip_size == None:
						return None
					return current.CreateChildAtOffset('[' + str(index) + ']',self.skip_size,self.data_type)
			else:
				print "foo"
				return None
		except Exception as err:
			print err
			return None

“foo” is printed when self.get_data_type() does not work. On my side, the right thing to do is add a fair amount of logging to better diagnose this and similar scenarios.

On your side, there are several possibilities:
(a) wait for the logging to be in, and then repeat the test case and send the new log files
(b) send the a.out and associated dSYM and let’s check if they look correct or the older compiler is doing something wrong with the debug info
(c) compile a new clang from TOT and retry using the updated compiler

Thanks,

Enrico Granata
:email: egranata@.com
✆ (four oh eight) 862-7683

Hi Enrico,

The old compiler was the culprit, it seems. With clang ToT the test passes.
I'm sending the a.out and its dSYM file anyway since this may be a bug in lldb anyway (it doesn't work with the current Mac OS X provided clang).

Thanks,

  Filipe

a.out.dSYM.zip (81.9 KB)

a.out (120 KB)

Hi Filipe,
glad to hear it works on a recent clang.
My bet is that the culprit is actually the compiler, not lldb.

The way the data formatter works is by accessing the type information for a specific child and extracting type information for the map out of it. This is what the DWARF emitted by old clang has to say about the type we use:

0x0000bff4: TAG_class_type [35]
AT_name( “__tree_node” )
AT_declaration( 0x01 )

0x0000bffa: TAG_class_type [35]
AT_name( “__tree_node_base” )
AT_declaration( 0x01 )

Not much. OTOH, the newer clang:

0x0000c6cc: TAG_class_type [13] *
AT_name( “__tree_node<std::__1::pair<int, int>, void *>” )
AT_byte_size( 0x28 )
AT_decl_file( “/usr/lib/c++/v1/__tree” )
AT_decl_line( 595 )

0x0000c6d5: TAG_inheritance [31]
AT_type( {0x00009b4a} ( __tree_node_base<void *> ) )
AT_data_member_location( +0 )
AT_accessibility( DW_ACCESS_public )

0x0000c6de: TAG_typedef [14]
AT_type( {0x0000d9bd} ( pair<int, int> ) )
AT_name( “value_type” )
AT_decl_file( “/usr/lib/c++/v1/__tree” )
AT_decl_line( 600 )

0x0000c6ea: TAG_member [15]
AT_name( “_value” ) <---- this is what we use, and the type information is good enough
AT_type( {0x0000c6de} ( value_type ) )
AT_decl_file( “/usr/lib/c++/v1/__tree” )
AT_decl_line( 602 )
AT_data_member_location( +28 )
AT_accessibility( DW_ACCESS_public )

0x0000c6fa: TAG_subprogram [17] *
AT_name( “__tree_node” )
AT_decl_file( “/usr/lib/c++/v1/__tree” )
AT_decl_line( 611 )
AT_declaration( 0x01 )
AT_external( 0x01 )
AT_accessibility( DW_ACCESS_public )
AT_explicit( 0x01 )

0x0000c706: TAG_formal_parameter [8]
AT_type( {0x00016c30} ( __tree_node<std::__1::pair<int, int>, void > ) )
AT_artificial( 0x01 )

0x0000c70c: TAG_formal_parameter [18]
AT_type( {0x00016c3a} ( const value_type& ) )

0x0000c711: NULL

0x0000c712: TAG_template_type_parameter [10]
AT_type( {0x0000d9bd} ( pair<int, int> ) )
AT_name( “_Tp” )

0x0000c71b: TAG_template_type_parameter [10]
AT_type( {0x0000d646} ( * ) )
AT_name( “_VoidPtr” )

0x0000c724: NULL

The type information for _value refers to:

0x0000d9bd: TAG_class_type [6] *
AT_name( “pair<int, int>” )
AT_byte_size( 0x08 )
AT_decl_file( “/usr/lib/c++/v1/utility” )
AT_decl_line( 212 )

which is the right type, and hence the test works :slight_smile:

Without type information there is not much that we can do. We could, of course, try and make up the type manually by string concatenation - that is done in the libstdcpp map formatter, but this latter approach is much safer when it works. If you look into the other formatter, you’ll see we need to play some tricks to keep things working:

we need this function as a temporary workaround for rdar://problem/10801549

which prevents us from extracting the std::pair<K,V> SBType out of the template

arguments for _Rep_Type _M_t in the map itself - because we have to make up the

typename and then find it, we may hit the situation were std::string has multiple

names but only one is actually referenced in the debug information. hence, we need

to replace the longer versions of std::string with the shorter one in order to be able

to find the type name

def fixup_class_name(self, class_name):
logger = Logger.Logger()
if class_name == ‘std::basic_string<char, std::char_traits, std::allocator >’:
return ‘std::basic_string’,True
if class_name == ‘basic_string<char, std::char_traits, std::allocator >’:
return ‘std::basic_string’,True
if class_name == ‘std::basic_string<char, std::char_traits, std::allocator >’:
return ‘std::basic_string’,True
if class_name == ‘basic_string<char, std::char_traits, std::allocator >’:
return ‘std::basic_string’,True
return class_name,False

def update(self):
logger = Logger.Logger()
try:
self.count = None

we will set this to True if we find out that discovering a node in the map takes more steps than the overall size of the RB tree

if this gets set to True, then we will merrily return None for any child from that moment on

self.garbage = False
self.Mt = self.valobj.GetChildMemberWithName(‘_M_t’)
self.Mimpl = self.Mt.GetChildMemberWithName(‘_M_impl’)
self.Mheader = self.Mimpl.GetChildMemberWithName(‘_M_header’)

map_type = self.valobj.GetType()
if map_type.IsReferenceType():
logger >> “Dereferencing type”
map_type = map_type.GetDereferencedType()

map_arg_0 = str(map_type.GetTemplateArgumentType(0).GetName())
map_arg_1 = str(map_type.GetTemplateArgumentType(1).GetName())

logger >> "map has args " + str(map_arg_0) + " and " + str(map_arg_1)

map_arg_0,fixed_0 = self.fixup_class_name(map_arg_0)
map_arg_1,fixed_1 = self.fixup_class_name(map_arg_1)

logger >> "arg_0 has become: " + str(map_arg_0) + " (fixed: " + str(fixed_0) + “)”
logger >> "arg_1 has become: " + str(map_arg_1) + " (fixed: " + str(fixed_1) + “)”

HACK: this is related to the above issue with the typename for std::string

being shortened by clang - the changes to typename display and searching to honor

namespaces make it so that we go looking for std::pair<const std::basic_string, …>

but when we find a type for this, we then compare it against the fully-qualified

std::pair<const std::basic_string<char, std::char_traits… and of course fail

the way to bypass this problem is to avoid using the std:: prefix in this specific case

if fixed_0 or fixed_1:
map_arg_type = "pair<const " + map_arg_0 + ", " + map_arg_1
else:
map_arg_type = "std::pair<const " + map_arg_0 + ", " + map_arg_1

if map_arg_1[-1] == ‘>’:
map_arg_type = map_arg_type + " >"
else:
map_arg_type = map_arg_type + “>”

logger >> "final contents datatype is: " + str(map_arg_type)

self.data_type = self.valobj.GetTarget().FindFirstType(map_arg_type)

logger >> "and the SBType is: " + str(self.data_type)

from libstdc++ implementation of _M_root for rbtree

self.Mroot = self.Mheader.GetChildMemberWithName(‘_M_parent’)
self.data_size = self.data_type.GetByteSize()
self.skip_size = self.Mheader.GetType().GetByteSize()
except:
pass

This is way more complicated than desirable. There should be ways to mark the test as an expected failure for older-than-X clang versions. Maybe that is the best way to tackle this issue.

Thanks.

Enrico Granata
:email: egranata@.com
✆ (four oh eight) 862-7683

Hi Enrico,

Indeed, lldb didn't have much to work with. At least we now know why.
I'll run the tests for my GIL patch, then and ping back when it's ready.

Regards,

  Filipe