Using data formatters to display QString

Hi,

I am trying to view the content of a QString, in gdb I could simply type from the command line:

(gdb)print mystr.toUtf8().constData()

and I would get the info, can I do something similar with lldb’s data formatters (hint: I am getting parse error)?

for example:
type summary add QString --summary-string “${var.toUtf8().constData()}”

​So my question is: can I use functions when defining ‘type summary’ or am I limited to variables members only?

Thanks,​

You can’t use expressions in summary strings.
We have thought about this several times and have a couple ideas on how it could be done but for now it’s not there.

If you need to resort to an expression, you can use a python formatter instead and then you are free to call as many expressions as you like.

However, this will cause a slowdown - running expressions is not free - and if you ever need to make sure nothing is altering your program state, running expressions might not be a safe bet.
Is there really no other way to get to those UTF8 bytes?

Enrico Granata <egranata@🍎.com>

QString stores UTF16 data inside.

28.04.2014, 19:56, “Enrico Granata” <egranata@apple.com>:

You can’t use expressions in summary strings.
We have thought about this several times and have a couple ideas on how it could be done but for now it’s not there.

If you need to resort to an expression, you can use a python formatter instead and then you are free to call as many expressions as you like.

However, this will cause a slowdown - running expressions is not free - and if you ever need to make sure nothing is altering your program state, running expressions might not be a safe bet.
Is there really no other way to get to those UTF8 bytes?

QString stores UTF16 data inside.

That is easy to deal with - refer to examples/summaries/unicode_strings.py in the LLDB source tree

All you want to do is take the utf16_summary function, and adjust it to reflect the innards of QString (what is the name of the child that contains the bytes, what is the name of the child that contains the length) - then you can tuck that function away in some file, and register it as a summary for QString - all without calling functions in the inferior process!


Regards,
Konstantin

- Enrico
:envelope_with_arrow: egranata@.com :phone: 27683

Enrico Granata wrote:

Eran Ifrah wrote:

You can’t use expressions in summary strings.

We have thought about this several times and have a couple ideas on how
it could be done but for now it’s not there.

If you need to resort to an expression, you can use a python formatter
instead and then you are free to call as many expressions as you like.

However, this will cause a slowdown - running expressions is not
free - and if you ever need to make sure nothing is altering your
program state, running expressions might not be a safe bet.
Is there really no other way to get to those UTF8 bytes?

QString is stored in UTF16 internally. It can be accessed directly
through structure member access and pointer arithmetic and converted
using Python. “Running expressions” is not needed.

Andre’

Here’s a small example for general reference:
Assume I have the following data structure:

#include
#include

class UTF16String {
public:
UTF16String (const char16_t data) {
len = std::char_traits<char16_t>::length(data);
str_data.reset(new char16_t[len]);
memcpy(str_data.get(),data,sizeof(char16_t)
(len+1));
}

private:
std::unique_ptr<char16_t> str_data;
size_t len;
};

int main() {
UTF16String string {u"Just some data in UTF16 here"};
return 0;
}

This is what it looks like “raw” on OS X:
(UTF16String) string = {
str_data = {
_ptr = {
std::__1::__libcpp_compressed_pair_imp<char16_t *, std::__1::default_delete<char16_t> > = {
_first = 0x00000001001037e0
}
}
}
len = 28
}

To define a formatter for it you essentially want to grab two elements: the data pointer (_first = 0x00000001001037e0) and the length (len = 30)
In our example, the length is defined in UTF16-characters rather than bytes. This is something you want to know when writing a formatter

So let’s delve right in:

def utf16string_summary(value,*rest):
str_data = value.GetChildMemberWithName(“str_data”).GetChildMemberWithName(“_ptr”).GetChildMemberWithName(“_first”)
length_vo = value.GetChildMemberWithName(“len”)

Now we have SBValues for the string data and for the length - we want the “number stored inside” the length:

length = length_vo.GetValueAsUnsigned(0)
if length == 0:
return ‘“”’

As a special case - if the length is zero, just return an empty string. I am not going to go in detail over all the possible checks here (hint: what if str_data’s value is zero?)

Now let’s grab the bytes - we want length char16_t at the location pointed to by our str_data:

data = str_data.GetPointeeData(0,length)

And now let’s grab a Python string out of those bytes:

error = lldb.SBError()
bytes = data.ReadRawData(error,0,2*length)

The 2*length argument is carefully crafted to ensure we get all the bytes we need - it of course depends on sizeof(char16_t) == 2

Python is pretty good at string management, all we have to do now is tell it to turn our string from UTF16 into UTF8:

return ‘“%s”’ % (bytes.decode(‘utf-16’).encode(‘utf-8’))

You’re done. To add the summary automatically to LLDB whenever you command script import the python file with the formatter:

def __lldb_init_module(debugger,*rest):
summary = lldb.SBTypeSummary.CreateWithFunctionName(“qstring.utf16string_summary”)
summary.SetOptions(lldb.eTypeOptionHideChildren)
debugger.GetDefaultCategory().AddTypeSummary(lldb.SBTypeNameSpecifier(“UTF16String”,False),summary)

This is what you get with the summary enabled:
(UTF16String) string = "Just some data in UTF16 here”

Find the C++ and the Python parts of the example attached for reference - and of course feel free to ping back with any additional questions

qstring.py (752 Bytes)

qstring.cpp (411 Bytes)

Hi Enrico and all,
Thanks for the references, it helped me a lot.
I am now able to view QString within lldb from the command line, but not from within my plugin :stuck_out_tongue: (this was my intention all the way)

I created the following 2 files:
~/.lldbinit, with this single line:

command script import /home/eran/.lldb/qstring.py

In the script: ~/.lldb/qstring.py, I placed the following content (a slightly modified printer based on your example):

import lldb

def utf16string_summary(value, rest):
f = open(‘/tmp/py.log’,‘w+b’)
f.write(‘inside utf16string_summary\n’)
f.close()
str_data = value.GetChildMemberWithName(“d”).GetChildMemberWithName(“data”)
length_vo = value.GetChildMemberWithName(“d”).GetChildMemberWithName(“size”)
length = length_vo.GetValueAsUnsigned(0)
if length == 0:
return ‘“”’
data = str_data.GetPointeeData(0, length)
error = lldb.SBError()
bytes = data.ReadRawData(error, 0, 2
length)
return ‘“%s”’ % (bytes.decode(‘utf-16’).encode(‘utf-8’))

def __lldb_init_module(debugger, *rest):
summary = lldb.SBTypeSummary.CreateWithFunctionName(“qstring.utf16string_summary”)
summary.SetOptions(lldb.eTypeOptionHideChildren)
debugger.GetDefaultCategory().AddTypeSummary( lldb.SBTypeNameSpecifier(“QString”, False), summary )

This setup works when I am using lldb-3.5 from the command line (i.e. QString is displayed in the following format: m_stringMemeber = “some content”)

Now, this is how I set it up from within my plugin:

I tried both:
m_debugger = lldb::SBDebugger::Create(true); // source init files

and I have also tried this:

m_debugger = lldb::SBDebugger::Create();


lldb::SBCommandReturnObject ret;

m_debugger.GetCommandInterpreter().HandleCommand(“command source /home/eran/.lldbinit”, ret);

if ( !ret.Succeeded() ) {
// print error here if any
}

Both did not have any affect, i.e. when I view the content of QString in the IDE, I don’t see the summary as it should
Any hints?

Thanks

Hi Enrico and all,
Thanks for the references, it helped me a lot.
I am now able to view QString within lldb from the command line, but not from within my plugin :stuck_out_tongue: (this was my intention all the way)

I created the following 2 files:
~/.lldbinit, with this single line:

command script import /home/eran/.lldb/qstring.py

In the script: ~/.lldb/qstring.py, I placed the following content (a slightly modified printer based on your example):

import lldb

def utf16string_summary(value, rest):
f = open(‘/tmp/py.log’,‘w+b’)
f.write(‘inside utf16string_summary\n’)
f.close()
str_data = value.GetChildMemberWithName(“d”).GetChildMemberWithName(“data”)
length_vo = value.GetChildMemberWithName(“d”).GetChildMemberWithName(“size”)
length = length_vo.GetValueAsUnsigned(0)
if length == 0:
return ‘“”’
data = str_data.GetPointeeData(0, length)
error = lldb.SBError()
bytes = data.ReadRawData(error, 0, 2
length)
return ‘“%s”’ % (bytes.decode(‘utf-16’).encode(‘utf-8’))

def __lldb_init_module(debugger, *rest):
summary = lldb.SBTypeSummary.CreateWithFunctionName(“qstring.utf16string_summary”)
summary.SetOptions(lldb.eTypeOptionHideChildren)
debugger.GetDefaultCategory().AddTypeSummary( lldb.SBTypeNameSpecifier(“QString”, False), summary )

This setup works when I am using lldb-3.5 from the command line (i.e. QString is displayed in the following format: m_stringMemeber = “some content”)

How recent/non recent is lldb-3.5?
Did you try with trunk at all?

Now, this is how I set it up from within my plugin:

I tried both:
m_debugger = lldb::SBDebugger::Create(true); // source init files

and I have also tried this:

m_debugger = lldb::SBDebugger::Create();


lldb::SBCommandReturnObject ret;

m_debugger.GetCommandInterpreter().HandleCommand(“command source /home/eran/.lldbinit”, ret);

if ( !ret.Succeeded() ) {
// print error here if any
}

Both did not have any affect, i.e. when I view the content of QString in the IDE, I don’t see the summary as it should
Any hints?

Well, I see two - or rather three - potential issues
First of all, Is the summary loaded? To check if it is, run “type summary list” and make sure you see a formatter for QString listed in the default category - and that the default category is enabled
Second, is the summary working? If it is loaded, we can test it manually: do something like this while you’re stopped in a frame with a QString variable
(lldb) script value = lldb.frame.FindVariable(“myQStringThingNameHere”)
(lldb) script print qstring.utf16string_summary(value,None)

Does anything come out with a manual invocation?

Assuming the summary is registered, and manually calling it works - how are you fetching the summary in your plugin?
Since it works at the command line, I assume the type name for QString is just plain QString with no namespaces in front. Is that correct?

Thanks

- Enrico
:envelope_with_arrow: egranata@.com :phone: 27683

Hi Enrico and all,
Thanks for the references, it helped me a lot.
I am now able to view QString within lldb from the *command line*, but
not from within my plugin :stuck_out_tongue: (this was my intention all the way)

I created the following 2 files:
~/.lldbinit, with this single line:

command script import /home/eran/.lldb/qstring.py

In the script: ~/.lldb/qstring.py, I placed the following content (a
slightly modified printer based on your example):

import lldb

def utf16string_summary(value, *rest):
    f = open('/tmp/py.log','w+b')
    f.write('inside utf16string_summary\n')
    f.close()
    str_data =
value.GetChildMemberWithName("d").GetChildMemberWithName("data")
    length_vo =
value.GetChildMemberWithName("d").GetChildMemberWithName("size")
    length = length_vo.GetValueAsUnsigned(0)
    if length == 0:
        return '""'
    data = str_data.GetPointeeData(0, length)
    error = lldb.SBError()
    bytes = data.ReadRawData(error, 0, 2*length)
    return '"%s"' % (bytes.decode('utf-16').encode('utf-8'))

def __lldb_init_module(debugger, *rest):
    summary =
lldb.SBTypeSummary.CreateWithFunctionName("qstring.utf16string_summary")
    summary.SetOptions(lldb.eTypeOptionHideChildren)
    debugger.GetDefaultCategory().AddTypeSummary(
lldb.SBTypeNameSpecifier("QString", False), summary )

This setup works when I am using lldb-3.5 from the command line (i.e.
QString is displayed in the following format: m_stringMemeber = "some
content")

How recent/non recent is lldb-3.5?
Did you try with trunk at all?

On this VirtualBox, ​I am using the Ubuntu 14.04 packages (the package name
is lldb-3.5-dev, but I am not sure of which revision is it...)

Now, this is how I set it up from within my plugin:

I tried both:
m_debugger = lldb::SBDebugger::Create(true); // source init files

and I have also tried this:

m_debugger = lldb::SBDebugger::Create();
...
lldb::SBCommandReturnObject ret;
m_debugger.GetCommandInterpreter().HandleCommand("command source
/home/eran/.lldbinit", ret);
if ( !ret.Succeeded() ) {
    // print error here if any
}

Both did not have any affect, i.e. when I view the content of QString in
the IDE, I don't see the summary as it should
Any hints?

Well, I see two - or rather three - potential issues
First of all, Is the summary loaded?

​Yes, it does.
After the debug session I added a call for debug purposes:
m_debugger.GetCommandInterpreter().HandleCommand("type summary list", ret);
And here are the relevant parts:

​[ 22:55:49:193 DBG ] codelite-lldb: type summary list returned:
[ 22:55:49:193 DBG ] -----------------------
[ 22:55:49:193 DBG ] Category: default (enabled)
[ 22:55:49:193 DBG ] -----------------------
[ 22:55:49:193 DBG ] QString: (not cascading)
[ 22:55:49:193 DBG ] wxPoint: `x = ${var.x}, y = ${var.y}`
[ 22:55:49:193 DBG ] wxRect: `(x = ${var.x}, y = ${var.y}) (width =
${var.width}, height = ${var.height})`
[ 22:55:49:193 DBG ] wxString: `${var.m_impl._M_dataplus._M_p}`
[ 22:55:49:193 DBG ] -----------------------
[ 22:55:49:193 DBG ] Category: objc (enabled)
[ 22:55:49:193 DBG ] -----------------------

​The first one "QString" was added using python script, the following 3
(wxPoint, wxString and wxRect) were added using 'type summary add..' simple
command all 3 are working (from within my plugin). The fact that the wx*
summaries are working properly within the plugin give me confident that the
method I am using to retrieve the data is correct.

To check if it is, run “type summary list” and make sure you see a

formatter for QString listed in the default category - and that the default
category is enabled
Second, is the summary working? If it is loaded, we can test it manually:
do something like this while you’re stopped in a frame with a QString
variable
(lldb)
​​
script value = lldb.frame.FindVariable(“myQStringThingNameHere”)
(lldb)
​​
script print qstring.utf16string_summary(value,None)

​Arg, I don't have a 'console' where I can free type commands ( I will

definitely add one to the plugin)​
For now, I added the above commands manually for debug purposes and here
are the results:

[ 23:13:24:029 DBG ] codelite-lldb: output of command 'script print
qstring.utf16string_summary(value, None)':
[ 23:13:24:029 DBG ] Traceback (most recent call last):
[ 23:13:24:029 DBG ] File "<input>", line 1, in <module>
[ 23:13:24:029 DBG ] NameError: name 'qstring' is not defined

The log clearly shows that it does not know what 'qstring' is... but what?

Does anything come out with a manual invocation?

Assuming the summary is registered, and manually calling it works - how
are you fetching the summary in your plugin?

​I don't think that this is the case as it works for other summaries (e.g.
wxString)

Since it works at the command line, I assume the type name for QString is
just plain QString with no namespaces in front. Is that correct?

​Yes, the same code (with the same ~/.lldbinit) running from command line:

(lldb)
Process 21809 stopped
* thread #1: tid = 21809, 0x0000000000400ae5 TestQString`main(argc=1,
argv=0x00007fff922a8688) + 184 at main.cpp:11, name = 'TestQString', stop
reason = step over
    frame #0: 0x0000000000400ae5 TestQString`main(argc=1,
argv=0x00007fff922a8688) + 184 at main.cpp:11
   8 Q_UNUSED( argv );
   9 QString str;
   10 str = QString("Hello %1").arg("world");
-> 11 printf("%s\n", str.toLocal8Bit().constData());
   12 return 0;
   13 }
(lldb) p str
(QString) $0 = "Hello world"

Note that the 'lldb' from the command line is was also installed from the
Ubuntu package lldb-3.5-dev, so we can also rule that one out...

Well that seems to imply we don’t know what the qstring module is when trying to use it to run the formatter

How are you loading the python script from your plugin?

Another thing that you could try is debug LLDB and see what happens when we try fetching the summary. I am expecting we will fail to find the function object and then we can take it from there.

Enrico Granata <egranata@🍎.com>

Hi,

I have tried to follow the Driver.cpp file (lldb main?) code and see what it does that I am not doing properly.

The only difference that I could tell was that it creates the debugger like this:

m_debugger( SBDebugger::Create(false) );

and later it manually does this:

sb_interpreter.SourceInitFileInHomeDirectory(result);

So I tried to use this approach as well - it failed (for me calling SourceInitFileInHomeDirectory did not work at all, i.e. I could not see QString registered)

However, sourcing the ~/.lldbinit file after the target creation did the trick:

wxString source_command;
source_command << “command source '” << ::wxGetHomeDir() << “/.lldbinit” << “'”;
DoExecutueShellCommand( source_command, true );

worked!

I can now view QString as pure strings in my plugin

Thanks for the help

Eran