Everyone: please download, build and test the new iohandler branch

svn co https://$USER@llvm.org/svn/llvm-project/lldb/branches/iohandler

The first step is to get it building for your platform as I am sure the Makefiles are out of date.

This branch will soon be merged back into top of tree, but I wanted to give all the major platforms time to submit patches against this to get things working on all systems before any buildbots get broken.

The major benefits include:
- editline is not built into the lldb shared library so all IOHandler objects can use the editline functionality.
- autocomplete now working in the embedded python interpreter
- history now working in the embedded python interpreter
- autocomplete now working for multi-line command entering (like in "breakpoint command add")
- when editing multiple lines you can use the UP and DOWN arrow keys to edit previous lines. This makes multi-line expressions and commands much easier to write and edit. Use ^B and ^N for next/prev history when in multi-line mode.
- curses is now supported with the new IOHandler infrastructure. To try this out, run and hit a breakpoint, and type "gui" on the command line to drop into the curses GUI mode! Lots of stuff isn't hooked up yet, but I am sure the open source community can help fill in some new views and improve existing ones.

So please get this building and test this on your system and let us know what issues you run into.

Greg Clayton

svn co https://$USER@llvm.org/svn/llvm-project/lldb/branches/iohandler

The first step is to get it building for your platform as I am sure the Makefiles are out of date.

Here's my report for FreeBSD. I haven't looked into any of the issues
in detail yet.

I always use cmake to build LLDB. For this branch I needed the
following changes:

source/API/CMakeLists.txt
- SBInputReader.cpp

source/Commands/CMakeLists.txt
+ CommandObjectGUI.cpp

source/Core/CMakeLists.txt
- InputReader.cpp
- InputReaderEZ.cpp
- InputReaderStack.cpp
+ IOHandler.cpp

source/Host/common/CMakeLists.txt
+ Editline.cpp

tools/driver/CMakeLists.txt
- IOChannel.cpp

(I can commit these changes to the iohandler branch if you like.)

In addition ::getcurx() and such resulted in a compilation failure --
on FreeBSD ncurses.h provides macro implementations of these. The man
page has this note:

       All of these interfaces are provided as macros and functions. The
       macros are suppressed (and only the functions provided) when
       NCURSES_OPAQUE is defined.

- autocomplete now working in the embedded python interpreter

When trying to run the interpreter to test out this I got:

(lldb) script
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tank/emaste/src/llvm/build-nodebug/lib/python2.7/site-packages/lldb/embedded_interpreter.py",
line 58
    def one_line (self, input):
    ^
IndentationError: unexpected indent

- curses is now supported with the new IOHandler infrastructure. To try this out, run and hit a breakpoint, and type "gui" on the command line to drop into the curses GUI mode! Lots of stuff isn't hooked up yet, but I am sure the open source community can help fill in some new views and improve existing ones.

Nifty - I can start the curses UI, but it seems none of the F-keys work.

I also have 19 failing tests when I run the test suite on this branch.

svn co https://$USER@llvm.org/svn/llvm-project/lldb/branches/iohandler

The first step is to get it building for your platform as I am sure the Makefiles are out of date.

Here's my report for FreeBSD. I haven't looked into any of the issues
in detail yet.

I always use cmake to build LLDB. For this branch I needed the
following changes:

source/API/CMakeLists.txt
- SBInputReader.cpp

source/Commands/CMakeLists.txt
+ CommandObjectGUI.cpp

source/Core/CMakeLists.txt
- InputReader.cpp
- InputReaderEZ.cpp
- InputReaderStack.cpp
+ IOHandler.cpp

source/Host/common/CMakeLists.txt
+ Editline.cpp

tools/driver/CMakeLists.txt
- IOChannel.cpp

(I can commit these changes to the iohandler branch if you like.)

Yes, please do!

In addition ::getcurx() and such resulted in a compilation failure --
on FreeBSD ncurses.h provides macro implementations of these. The man
page has this note:

      All of these interfaces are provided as macros and functions. The
      macros are suppressed (and only the functions provided) when
      NCURSES_OPAQUE is defined.

Sounds good, please remove the "::" so things compile.

- autocomplete now working in the embedded python interpreter

When trying to run the interpreter to test out this I got:

(lldb) script
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tank/emaste/src/llvm/build-nodebug/lib/python2.7/site-packages/lldb/embedded_interpreter.py",
line 58
   def one_line (self, input):
   ^
IndentationError: unexpected indent

Fixed:

% svn commit
Sending source/Interpreter/embedded_interpreter.py
Transmitting file data .
Committed revision 198452.

- curses is now supported with the new IOHandler infrastructure. To try this out, run and hit a breakpoint, and type "gui" on the command line to drop into the curses GUI mode! Lots of stuff isn't hooked up yet, but I am sure the open source community can help fill in some new views and improve existing ones.

Nifty - I can start the curses UI, but it seems none of the F-keys work.

This is something you will want to figure out on your system to see if you can get the function keys working. There is a work around for Apple builds where we look for \033OP for F1, so if you search for "\033OP" in the sources you can see the work around that we used to map non-standard function keys to the correct key definition.

If you type "infocmp" at your unix command line you should get a dump of the current key mappings and you should see what your function keys are mapped to. On my system I see:

kf1=\E[11~
kf2=\E[12~
etc...

If you place a breakpoint on the following line (IOHandler.cpp:1686):

                    HandleCharResult key_result = m_window_sp->HandleChar(ch);

then press F1 on your keyboard and see what characters come through, you can see what your function key is coming through as. It is probably an escape character followed by a few keys. Then you can either figure out how to enable function keys on your version of curses, or handle the keys like I did for Mac in the #if defined (__APPLE__) code that intercepts and remaps keys.

As a work around, you can use the right and left arrow keys to bring up the menus so you can select LLDB->Exit.

Most menus don't work at the moment except for "LLDB->Exit", but I will be hooking them up soon.

A few things to try in "gui" mode:

Press TAB to change the active window. The active window has the solid white border around it. Each window can handle keys differently.

when the source view is selected press the following keys:

s - step in
n - step over
o - step out
k - kill
d - detach
b - set breakpoint on selected line
UP/DOWN - move line selection around so you can set breakpoints and run to that line
SPACE - set one shot breakpoint on selected line and and run to that line
, = page up
. = page down

When the "variables" view is selected:
UP/DOWN - use the arrow keys to change the selection around
RIGHT - expand current item
LEFT - unexpand if the item is expandable, else select the parent if not expandable
x - format value as hex
u = format value as unsigned
o = format value as octal
, = page up
. = page down

I also have 19 failing tests when I run the test suite on this branch.

Yep, I have about the same thing and we will need to work these things out and fix them.

Greg

(I can commit these changes to the iohandler branch if you like.)

Yes, please do!

I've committed this and the getcurx and such global scoping change just now.

Fixed:

% svn commit
Sending source/Interpreter/embedded_interpreter.py
Transmitting file data .
Committed revision 198452.

Thanks - a new failure now:

(lldb) script
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/data/emaste/src/llvm/build/lib/python2.7/site-packages/lldb/embedded_interpreter.py",
line 60, in run_python_interpreter
    repl = SimpleREPL('>>> ', dict)
NameError: global name 'SimpleREPL' is not defined

This is something you will want to figure out on your system to see if you can get the function keys working. There is a work around for Apple builds where we look for \033OP for F1, so if you search for "\033OP" in the sources you can see the work around that we used to map non-standard function keys to the correct key definition.

Ah, so it turns out it actually works as is on my FreeBSD-CURRENT
laptop. (I observed the failure on my desktop, which has an older
version.)

A few things to try in "gui" mode:

Press TAB to change the active window. The active window has the solid white border around it. Each window can handle keys differently.

when the source view is selected press the following keys:

...
SPACE - set one shot breakpoint on selected line and and run to that line

Stepping looks like it works fine but SPACE didn't do anything for me.

When the "variables" view is selected:
UP/DOWN - use the arrow keys to change the selection around

One issue I noticed - there seems to be an "off by two" in the
variable window - I can move the highlight down twice from the last
displayed row before the list starts scrolling.

I also have 19 failing tests when I run the test suite on this branch.

Yep, I have about the same thing and we will need to work these things out and fix them.

Ok - good to know it's not (necessarily) a FreeBSD-specific issue.

-Ed

(I can commit these changes to the iohandler branch if you like.)

Yes, please do!

I've committed this and the getcurx and such global scoping change just now.

Great!

Fixed:

% svn commit
Sending source/Interpreter/embedded_interpreter.py
Transmitting file data .
Committed revision 198452.

Thanks - a new failure now:

(lldb) script
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/data/emaste/src/llvm/build/lib/python2.7/site-packages/lldb/embedded_interpreter.py",
line 60, in run_python_interpreter
   repl = SimpleREPL('>>> ', dict)
NameError: global name 'SimpleREPL' is not defined

Fixed:

% svn commit
Sending source/Interpreter/embedded_interpreter.py
Transmitting file data .
Committed revision 198629.

This is something you will want to figure out on your system to see if you can get the function keys working. There is a work around for Apple builds where we look for \033OP for F1, so if you search for "\033OP" in the sources you can see the work around that we used to map non-standard function keys to the correct key definition.

Ah, so it turns out it actually works as is on my FreeBSD-CURRENT
laptop. (I observed the failure on my desktop, which has an older
version.)

Good.

A few things to try in "gui" mode:

Press TAB to change the active window. The active window has the solid white border around it. Each window can handle keys differently.

when the source view is selected press the following keys:

...
SPACE - set one shot breakpoint on selected line and and run to that line

Stepping looks like it works fine but SPACE didn't do anything for me.

That should have been ENTER, not SPACE...

When the "variables" view is selected:
UP/DOWN - use the arrow keys to change the selection around

One issue I noticed - there seems to be an "off by two" in the
variable window - I can move the highlight down twice from the last
displayed row before the list starts scrolling.

This was fixed over the weekend. Try it again! Also try the new registers window using View->Registers in the menu!

I also have 19 failing tests when I run the test suite on this branch.

Yep, I have about the same thing and we will need to work these things out and fix them.

Ok - good to know it's not (necessarily) a FreeBSD-specific issue.

I am going to fix all of the test suite issues that I can and I will let you know when to run the test suite again.

Greg

NameError: global name 'SimpleREPL' is not defined

Fixed:

% svn commit
Sending source/Interpreter/embedded_interpreter.py
Transmitting file data .
Committed revision 198629.

Thanks, that looks like the last issue and I can run the built-in
python again. I still see one oddity, but I believe this happened
prior to this branch - I have to hit ^C to get the (lldb) prompt back
after exiting python:

(lldb) script
Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.

quit()

load: 0.34 cmd: lldb-3.5 7279 [select] 60.15r 0.14u 0.09s 0% 79184k
^C(lldb)

(The "load" line is me hitting ^T while waiting.)

SPACE - set one shot breakpoint on selected line and and run to that line

Stepping looks like it works fine but SPACE didn't do anything for me.

That should have been ENTER, not SPACE...

Ahh. That works.

When the "variables" view is selected:
UP/DOWN - use the arrow keys to change the selection around

One issue I noticed - there seems to be an "off by two" in the
variable window - I can move the highlight down twice from the last
displayed row before the list starts scrolling.

This was fixed over the weekend. Try it again! Also try the new registers window using View->Registers in the menu!

Yep, that works now too, thanks!

I noticed now that the source window displays two extra characters (so
long lines overwrite the right border and the left border on the next
line). The register window is nice (it also overwrites the border
though).

I am going to fix all of the test suite issues that I can and I will let you know when to run the test suite again.

Great. Once it works on OS X I will track down remaining FreeBSD
failures, if any.

-Ed

The Windows build was broken on this branch, but with a little work I was able to fix that.

The included patch should remedy things (builds against revision 198602).

Also these two files need to be moved and renamed:
- "tools/driver/ELWrapper.cpp" should be moved and renamed to "source/host/windows/EditLineWin.cpp"
- "tools/driver/ELWrapper.h" should be moved and renamed to "include/host/windows/editlinewin.h"

Also I am a little worried about the use of unix named pipes in GDBRemoteCommunication.cpp and Process.cpp, since this isn't portable to windows.
What are these pipes for?
Is it something that can be harmlessly disabled on windows?

Overall this looks like a nice branch for LLDB and seems to clean a lot of things up.

Thanks,
Aidan Dodds

iohandler_win.patch (8.54 KB)

Hi Greg,

From: lldb-dev-bounces@cs.uiuc.edu [mailto:lldb-dev-bounces@cs.uiuc.edu]
On Behalf Of Greg Clayton
Sent: 02 January 2014 22:57
To: lldb-dev@cs.uiuc.edu
Subject: [lldb-dev] Everyone: please download, build and test the new
iohandler branch

svn co https://$USER@llvm.org/svn/llvm-project/lldb/branches/iohandler

The first step is to get it building for your platform as I am sure the Makefiles
are out of date.

This branch will soon be merged back into top of tree, but I wanted to give
all the major platforms time to submit patches against this to get things
working on all systems before any buildbots get broken.

The major benefits include:
- editline is not built into the lldb shared library so all IOHandler objects can
use the editline functionality.
- autocomplete now working in the embedded python interpreter
- history now working in the embedded python interpreter
- autocomplete now working for multi-line command entering (like in
"breakpoint command add")
- when editing multiple lines you can use the UP and DOWN arrow keys to
edit previous lines. This makes multi-line expressions and commands much
easier to write and edit. Use ^B and ^N for next/prev history when in multi-
line mode.
- curses is now supported with the new IOHandler infrastructure. To try this
out, run and hit a breakpoint, and type "gui" on the command line to drop
into the curses GUI mode! Lots of stuff isn't hooked up yet, but I am sure the
open source community can help fill in some new views and improve existing
ones.

So please get this building and test this on your system and let us know
what issues you run into.

I tried to build this branch using cmake on Ubuntu 12.04 with gcc 4.8.
There were some build issues that I fixed as follows.

Include limits.h in Editline.cpp for PATH_MAX and libncurses in the cmake files.
I also noted that libedit version on my system was missing
some defines like EL_PROMPT_ESC. I have to get hold of a recent version for libedit.
After that lldb builds fine on my system. Still have to test it though.

I am pasting the changes needed for the build below.

Index: source/Host/common/Editline.cpp

Great, please check in any changes you need in order to build!

I fixed the remaining issue on MacOSX:

% svn commit
Sending include/lldb/Core/Debugger.h
Sending include/lldb/Interpreter/PythonDataObjects.h
Sending include/lldb/Interpreter/ScriptInterpreter.h
Sending include/lldb/Interpreter/ScriptInterpreterPython.h
Sending source/Commands/CommandObjectProcess.cpp
Sending source/Core/Debugger.cpp
Sending source/Core/IOHandler.cpp
Sending source/Host/common/Editline.cpp
Sending source/Interpreter/ScriptInterpreterPython.cpp
Sending source/Plugins/Process/Utility/RegisterContextLLDB.cpp
Sending source/Plugins/Process/Utility/RegisterContextLLDB.h
Sending source/Plugins/Process/Utility/UnwindLLDB.cpp
Sending source/Plugins/Process/mach-core/ProcessMachCore.cpp
Sending source/Symbol/ClangASTType.cpp
Sending source/Target/Process.cpp
Sending test/functionalities/command_script/import/rdar-12586188/TestRdar12586188.py
Sending test/functionalities/command_source/TestCommandSource.py
Sending test/functionalities/conditional_break/.lldb
Sending test/functionalities/conditional_break/conditional_break.py
Sending test/functionalities/connect_remote/EchoServer.py
Sending test/functionalities/connect_remote/TestConnectRemote.py
Sending test/functionalities/inferior-assert/TestInferiorAssert.py
Sending test/python_api/default-constructor/TestDefaultConstructorForAPIObjects.py
Sending test/python_api/default-constructor/sb_debugger.py
Deleting test/python_api/default-constructor/sb_inputreader.py
Deleting test/python_api/input_reader
Transmitting file data ........................
Committed revision 198719.

Please try a test suite run on your end when you have a chance.

I fixed the remaining issue on MacOSX:

...

Please try a test suite run on your end when you have a chance.

Ok, I still see five failures on this branch on FreeBSD.

TestCommandRegex (timeout)
TestDataFormatterGlobals (segfault)
TestDataFormatterCategories (segfault)
TestInferiorAssert (missing -> for current instruction in disassembly)
TestAliases (differing expected failure text)

I'll have a quick look at these tomorrow, with the goal of having at
least the segfaults fixed it gets merged.

Great, please check in any changes you need in order to build!

Committed in revision 198773.

Hi all,

Running the iohandler branch on Ubuntu 12.04 using gcc 4.8.2, using configure rather than CMake, ran into a similar issue as Abid (needed to add -lcurses, and also add back in -ledit; patch attached). I’m getting these test failures:

Ran 272 tests.
Failing Tests (4)
FAIL: LLDB (suite) :: TestCommandRegex.py (Linux spucci-linux.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)
FAIL: LLDB (suite) :: TestDataFormatterCategories.py (Linux spucci-linux.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)
FAIL: LLDB (suite) :: TestDataFormatterGlobals.py (Linux spucci-linux.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)
FAIL: LLDB (suite) :: TestConvenienceVariables.py (Linux spucci-linux.mtv.corp.google.com 3.2.5-gg1336 #1 SMP Thu Aug 29 02:37:18 PDT 2013 x86_64 x86_64)

We will be looking at these shortly, but we’re also tracking a compilation error that’s blocking this now on this branch (more about that shortly).

  • Steve

Patch:

Index: lib/Makefile

iohandler-patch.txt (473 Bytes)

Patch looks good.

Okay - so since Abid is seeing it as well, I’m inclined to put in both (1) the makefile fix for non-cmake (i.e. configure-based) makes, and (2) the explicit operator=.

We’re tracing through an editline failure in a regex test at the moment. These are test failures, though, not build failures. It is probably worth it for me to get those two build break issues in, then, before we resolve all the test failures.

-Todd

Here is the final patch we need to get lldb compiling on the iochannel branch on Ubuntu 12.04 x86_64 using configure and makefiles:

Index: source/Core/IOHandler.cpp

spucci-2014-01-10.diff (1.29 KB)

There was a slight tweak to the change above to deal with modifications that just came in on IOHandler’s TreeItem.

Here is the change submission:

Sending lib/Makefile

Sending source/Core/IOHandler.cpp

Transmitting file data …

Committed revision 198951.

Final patch attached.

spucci-2014-01-10_r2.diff (1.22 KB)

Looks good!

Note even with this, we are still having a libedit issue on multiline input handling (as exposed through the regex command test). We’re looking at this now.

I believe I know what this is. More when I’ve confirmed…

  • Steve

We have a fix in the iohandler branch for a test failure of TestCommandRegex.py. When using libedit, the multi-line handler is stripping one too many lines from the collected input. This is causing ‘command regex’ and its related test to fail. The test was also expecting different output than the command currently emits.

Here is a patch for the proposed fix:

Index: test/functionalities/command_regex/TestCommandRegex.py

spucci-2014-01-10_03.diff (1.76 KB)