FreeBSD thread abort from pthread_cond_timedwait call within signal handler

I can now launch my inferior process on FreeBSD, but fail to get a
notification at process termination and then lldb waits indefinitely.
Upon hitting ^C I get an abort from an assertion in the thread library
"Fatal error 'thread was already on queue.'" It looks to come from
calling non-async-signal-safe pthread_cond_timedwait from the signal
handler:

#1 0x0000000805ef756c in abort () at
/home/emaste/src/9/lib/libc/stdlib/abort.c:65
#2 0x0000000800850d2c in _thread_exit (fname=<value optimized out>,
lineno=<value optimized out>, msg=<value optimized out>)
    at /home/emaste/src/9/lib/libthr/thread/thr_exit.c:182
#3 0x000000080084fc1d in cond_wait_common (cond=<value optimized

, mutex=<value optimized out>, abstime=<value optimized out>,

    cancel=<value optimized out>) at
/home/emaste/src/9/lib/libthr/thread/thr_cond.c:222
#4 0x00000008013d3995 in lldb_private::Condition::Wait
(this=0x7ffffedf5d20, mutex=@0x7ffffedf5d18, abstime=0x7ffffedf5c58,
    timed_out=0x7ffffedf5a1f) at
../tools/lldb/source/Host/common/Condition.cpp:89
#5 0x00000008013d213d in
lldb_private::Predicate<bool>::WaitForValueEqualTo
(this=0x7ffffedf5d10, value=true, abstime=0x7ffffedf5c58,
    timed_out=0x7ffffedf5a1f) at Predicate.h:338
#6 0x000000080143e105 in
lldb_private::Listener::WaitForEventsInternal (this=0x7ffffedf5cb8,
timeout=0x7ffffedf5c58, broadcaster=0x0,
    broadcaster_names=0x0, num_broadcaster_names=0, event_type_mask=0,
event_sp=@0x7ffffedf5ca8) at
../tools/lldb/source/Core/Listener.cpp:433
#7 0x000000080143e4b1 in lldb_private::Listener::WaitForEvent
(this=0x7ffffedf5cb8, timeout=0x7ffffedf5c58,
event_sp=@0x7ffffedf5ca8)
    at ../tools/lldb/source/Core/Listener.cpp:481
#8 0x0000000801747729 in lldb_private::Process::Halt
(this=0x806dbe800, clear_thread_plans=false)
    at ../tools/lldb/source/Target/Process.cpp:3389
#9 0x000000080174bdb8 in
lldb_private::Process::ProcessInputReaderCallback (baton=0x806dbe800,
reader=@0x806feef40,
    notification=lldb::eInputReaderInterrupt, bytes=0x0, bytes_len=0)
at ../tools/lldb/source/Target/Process.cpp:4540
#10 0x0000000801435cdc in lldb_private::InputReader::Notify
(this=0x806feef40, notification=lldb::eInputReaderInterrupt)
    at ../tools/lldb/source/Core/InputReader.cpp:349
#11 0x0000000801408901 in
lldb_private::Debugger::DispatchInputInterrupt (this=0x806c24800) at
../tools/lldb/source/Core/Debugger.cpp:866
#12 0x00000008012bc165 in lldb::SBDebugger::DispatchInputInterrupt
(this=0x7fffffffd3d8) at ../tools/lldb/source/API/SBDebugger.cpp:833
#13 0x0000000000410a44 in sigint_handler (signo=2) at
../tools/lldb/tools/driver/Driver.cpp:1672
#14 0x000000080084b62b in handle_signal (actp=<value optimized out>,
sig=<value optimized out>, info=<value optimized out>,
    ucp=<value optimized out>) at
/home/emaste/src/9/lib/libthr/thread/thr_sig.c:239
#15 0x000000080084b1bf in thr_sighandler (sig=2, info=<value optimized

, _ucp=<value optimized out>)

    at /home/emaste/src/9/lib/libthr/thread/thr_sig.c:182
#16 0x00007ffffffff003 in ?? ()
#17 0x000000080084b090 in sigaction () at
/home/emaste/src/9/lib/libthr/thread/thr_sig.c:580
#18 0x000000080084fad5 in cond_wait_common (cond=<value optimized

, mutex=0x806d646f0, abstime=0x0, cancel=1)

    at /home/emaste/src/9/lib/libthr/thread/thr_cond.c:243
#19 0x00000008013d39b2 in lldb_private::Condition::Wait
(this=0x806dbeaa8, mutex=@0x806dbeaa0, abstime=0x0,
timed_out=0x7ffffedf6bcf)
    at ../tools/lldb/source/Host/common/Condition.cpp:92
#20 0x00000008013d213d in
lldb_private::Predicate<bool>::WaitForValueEqualTo (this=0x806dbea98,
value=true, abstime=0x0,
    timed_out=0x7ffffedf6bcf) at Predicate.h:338
#21 0x000000080143e105 in
lldb_private::Listener::WaitForEventsInternal (this=0x806dbea40,
timeout=0x0, broadcaster=0x0, broadcaster_names=0x0,
    num_broadcaster_names=0, event_type_mask=0,
event_sp=@0x7ffffedf6e78) at
../tools/lldb/source/Core/Listener.cpp:433
#22 0x000000080143e4b1 in lldb_private::Listener::WaitForEvent
(this=0x806dbea40, timeout=0x0, event_sp=@0x7ffffedf6e78)
    at ../tools/lldb/source/Core/Listener.cpp:481
#23 0x000000080173ddcc in lldb_private::Process::WaitForEventsPrivate
(this=0x806dbe800, timeout=0x0, event_sp=@0x7ffffedf6e78,
    control_only=false) at ../tools/lldb/source/Target/Process.cpp:1423
#24 0x00000008017488e4 in lldb_private::Process::RunPrivateStateThread
(this=0x806dbe800) at ../tools/lldb/source/Target/Process.cpp:4004
#25 0x000000080174833d in lldb_private::Process::PrivateStateThread
(arg=0x806dbe800) at ../tools/lldb/source/Target/Process.cpp:3986
#26 0x00000008013cf29a in ThreadCreateTrampoline (arg=0x806fe5fa0) at
../tools/lldb/source/Host/common/Host.cpp:576
#27 0x000000080084722b in thread_start (curthread=0x806c0f000) at
/home/emaste/src/9/lib/libthr/thread/thr_create.c:284
#28 0x0000000000000000 in ?? ()

Yes, at present lldb isn't too careful to limit the work it does in the SIGINT handler. In this case, lldb is handling stdin for the target process, so the ^C should cause the target to be halted. The current code is actually trying to execute the Halt in the signal handler.

Note, the Driver's Interrupt IO handler tries to be a little better, calling Process::AsyncInterrupt which just sends an eBroadcastBitInterrupt event to the process. Note, however, that this actually requires a bunch more work than should be done in a signal handler (lock the event mutex, and new up an Event, putting it in a std::vector, etc.) But we might get away with that on FreeBSD more easily. I checked in that change, see if that gets you any further.

To fix this really pedantically, we'd have to add a thread that was watching for interrupts, so that AsyncInterrupt could message it with something safe like writing to a pipe the interrupt thread was selecting on, and it would in turn wake up the process. That's a bit more work than I have time to do on this right now, especially since we haven't had any reports of this causing problems in real life...

Jim

Thanks Jim, with that change I no longer abort upon hitting ^C, and if
I launch a 'sleep 60' hitting ^C brings me back to an lldb prompt.

If I back out some later fixes in my tree to reproduce the original
case and hit ^C lldb doesn't respond. I think this is just a side
effect of a bug in the FreeBSD ProcessMonitor though, where the
inferior has terminated already but lldb hasn't been notified.