Windows lldb prompt issue

I run lldb on Windows and Linux, launching my gdb-server based Hexagon simulator automatically on a process launch. On Windows I’m seeing the (lldb) prompt before the stop message, and no prompt after. On Linux I see the prompt after.

Windows:

Current executable set to ‘u:\lldb_test\factwin’ (hexagon).

(lldb) b main

Breakpoint 1: where = factwin`main + 28 at factorial.c:32, address = 0x00005130

(lldb) r

Process 1 launched: ‘u:\lldb_test\factwin’ (hexagon)

(lldb) Process 1 stopped

  • thread #1: tid = 0x0001, 0x00005130 factwin`main(argc=1, argv=0x0000e110) + 28 at factorial.c:32, stop reason = breakpoint 1.1

frame #0: 0x00005130 factwin`main(argc=1, argv=0x0000e110) + 28 at factorial.c:32

29 }

30 */

31

→ 32 base = 10;

33

34 printf(“Factorial of %d is %d\n”, base, factorial(base));

35 return 0;

Linux:

Current executable set to ‘/usr2/tedwood/lldb_test/factorial’ (hexagon).

(lldb) b main

Breakpoint 1: where = factorial`main + 28 at factorial.c:32, address = 0x00004130

(lldb) r

Process 1 launched: ‘/usr2/tedwood/lldb_test/factorial’ (hexagon)

Process 1 stopped

  • thread #1: tid = 0x0001, 0x00004130 factorial`main(argc=1, argv=0x0000b100) + 28 at factorial.c:32, stop reason = breakpoint 1.1

frame #0: 0x00004130 factorial`main(argc=1, argv=0x0000b100) + 28 at factorial.c:32

29 }

30 */

31

→ 32 base = 10;

33

34 printf(“Factorial of %d is %d\n”, base, factorial(base));

35 return 0;

(lldb)

Any idea why the prompt is coming out before the stop message?

This has been a longstanding issue which I just haven’t been able to prioritize over things such as PDB support, minidump support, test fixes, etc. There’s various thread synchronization stuff that happens between the IOHandler thread and the debugger thread, and I guess there’s a race condition or something.

Long distance high five if you can get a patch that solves this, I agree it’s really annoying.

All input and output are handled by IOHandler subclasses. There is a stack of them. When you start LLDB you get one thing on the IOHandler stack: the LLDB command interpreter:

IOHandlers:
[0] IOHandlerEditline for command interpreter

If you type "script" at the LLDB command prompt, the command interpreter pushes a IOHandlerPythonInterpreter onto the stack and all STDIN/OUT/ERR. Now your stack looks like:

[0] IOHandlerEditline for command interpreter
[1] IOHandlerPythonInterpreter

When you type "quit()" in the IOHandlerPythonInterpreter, it will exit and pop off and return to:

[0] IOHandlerEditline for command interpreter

Note that these transitions happen on the same thread, so it is very easy to coordinate the output since it all happens in one thread of execution.

When your process runs, by typing "run" on the command interpreter, the Process might push an IOHandler onto the stack so that when the process resumes you can do STDIN and receive any STDOUT/STDERR when your program runs, but it does so from _another_ thread. This is the only IOHandler that does things this way. So as soon as your resume a process that you have launched (if you attach, you don't get STDIN/OUT/ERR) we push the IOHandlerProcessSTDIO. When you stop, we must pop the IOHandlerProcessSTDIO. Again, this popping of the IOHandlerProcessSTDIO happens on another thread, not from the thread that is running the command interpreter. So this is why we have trouble coordinating the output. The threads the LLDB driver has are as follows:

1 - thread that runs the command interpreter
2 - thread that listens for debugger events (this includes process events for stops and runs)
3 - private process state thread that tracks any starts and stops and figure out when the send a public state change event

Thread 3 might start and stop a process 10 times for one source level single step, but it will only send out one public eStateRunning event and one eStateStopped event. So when a public eStateStopped event get delivered from thread 3 to thread 2, we set off a chain of events where we must pop the

So when a stopped event comes in, thread 2 receives the eStateStopped event and will handle flushing any pending STDOUT and STDERR and then pop the IOHandlerProcessSTDIO. When anyone wants to display text on the top IOHandler's output file, the must coordinate with the IOhandler stack. This is done by calling:

void
Debugger::PrintAsync (const char *s, size_t len, bool is_stdout);

This coordinates with the top IOHandler by calling:

void
IOHandler::PrintAsync (Stream *stream, const char *s, size_t len);

Now the command interpreter, when active will print its prompt of "(lldb) ". When async output comes in, it will eventually end up calling:

void
Editline::PrintAsync (Stream *stream, const char *s, size_t len)
{
    Mutex::Locker locker(m_output_mutex);
    if (m_editor_status == EditorStatus::Editing)
    {
        MoveCursor(CursorLocation::EditingCursor, CursorLocation::BlockStart);
        fprintf(m_output_file, ANSI_CLEAR_BELOW);
    }
    stream->Write (s, len);
    stream->Flush();
    if (m_editor_status == EditorStatus::Editing)
    {
        DisplayInput();
        MoveCursor(CursorLocation::BlockEnd, CursorLocation::EditingCursor);
    }
}

This clears the current line, so it erases the "(lldb) ", prints the output in "stream", the it must refresh the current IOHandler, which for out command interpreter will restore the "(lldb) " prompt.

So the question is: does the erase/print/restore feature in Editline::PrintAsync() even work on windows? I know that there is an Editline implementation on windows, but I am not sure it will do the right thing in this case as we are printing control characters to the screen in order to clear the "(lldb) " prompt with:

    if (m_editor_status == EditorStatus::Editing)
    {
        MoveCursor(CursorLocation::EditingCursor, CursorLocation::BlockStart);
        fprintf(m_output_file, ANSI_CLEAR_BELOW);
    }

Then we write the "Stream" content. So after a very long story, you might better understand what is happening. It would be worth testing something on windows where you just call Debugger::PrintAsync() while the "(lldb) " prompt is up and on the screen and see what happens. If this isn't working, then we know why the issue is happening. So if you have the following displayed in the LLDB driver:

"(lldb) "

And you call "Debugger::PrintAsync("hello world\n", 12, true)" on a thread that is not thread 1 from the initial thread list shown at the beginning, you should see your screen be:

"hello world
(lldb) "

If you see:

"(lldb)
hello world
(lldb) "

Then you know that the erasing feature isn't working in Editline::PrintAsync() and it will explain why you see this issue.

Greg Clayton

I dug into this some more. Windows is not using editline, so it's using the LLDB_DISABLE_LIBEDIT code path.

IOHandlerEditline::GetLine() will print the prompt, then get the next line. Type in a command, and it will continue doing this. The prompt comes out before the command finishes (in this case, anyway), so we get the prompt, then the output from the command, then nothing.

I've got a simple solution that works for hitting breakpoints and stepping, at least. In IOHandlerEditline::PrintAsync(), in the LLDB_DISABLE_LIBEDIT code path, print the prompt after calling IOHandler::PrintAsync().

I'll put up a patch to phabricator.

You might see if there is some way to implement an equivalent of IOHandlerEditline that works on the windows terminal? Maybe a class like IOHandlerWindowsTerminal which would be specified to native windows terminal. There has to be a way to control the cursor and do line kind of commands. I believe that erase to beginning of line is all you would need to do to fix this correctly.

Yea Windows does have methods for doing this, it’s just windows specific as you said and requires someone to actually do it.

Seems like there are library commands you can call: