lldb 340.4.119 unable to attach (El Capitan)

Hi,

I tried Jason Molenda’s test code on El Capitan, lldb-340.4.119 (Jason Molenda via lldb-dev | 3 Oct 02:59 2015).

I’m connected to a remote VM using ssh.

tst$ echo ‘int main () { }’ > /tmp/a.c

tst$ xcrun clang /tmp/a.c -o /tmp/a.out

tst$ xcrun lldb /tmp/a.out

(lldb) target create “/tmp/a.out”

Current executable set to ‘/tmp/a.out’ (x86_64).

(lldb) r

error: process exited with status -1 (unable to attach)

(lldb) run

error: process exited with status -1 (unable to attach)

(lldb) quit

tst$ ps -ef|grep a.out

502 33174 1 0 12:20PM ttys000 0:00.00 /tmp/a.out

502 33187 1 0 12:20PM ttys000 0:00.00 /tmp/a.out

I can’t even kill those using kill -9.

What’s going on here?

I tried the above because in fact I had a process which a segmentation fault 11, here’s what lldb makes out of the core dump:

txt$ lldb /cores/core.33158

(lldb) target create “/cores/core.33158”

warning: (x86_64) /cores/core.33158 load command 175 LC_SEGMENT_64 has a fileoff

  • filesize (0x31c57000) that extends beyond the end of the file (0x31c56000), t

he segment will be truncated to match

warning: (x86_64) /cores/core.33158 load command 176 LC_SEGMENT_64 has a fileoff

(0x31c57000) that extends beyond the end of the file (0x31c56000), ignoring thi

s section

Current executable set to ‘/cores/core.33158’ (x86_64).

(lldb)

Is this related? Compiled with g++ :

tst$ g++ --version

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-inc

lude-dir=/usr/include/c++/4.2.1

Apple LLVM version 7.0.2 (clang-700.1.81)

Target: x86_64-apple-darwin15.0.0

Thread model: posix

Thx,

Andre

Hi Andre,

For what it's worth, this is often a harmless warning message when debugging a user process core dump. The core creator code in the kernel adds an extra memory segment to the core file when it writes it out. There's a bug report tracking the issue but it's pretty much cosmetic so it hasn't been addressed yet. Try debugging your core file and see if it works. You may want to specify the name of your binary on the lldb cmd line like 'lldb a.out -c /tmp/core.33158'.

J

Thanks Todd for your hints.

I did’t compile stuff myself, all tools came from Apple’s Command_Line_Tools_OS_X_10.11_for_Xcode_7.2.dmg which I installed on a “fresh” El Capitan VM. I have requested more info about where this image came from, but the guys are on holiday.
Here’s a bit of the install history:
sudo installer -store -pkg "Command Line Tools (OS X 10.11).pkg" -target /

Installer output:

installer: Note: running installer as an admin user (instead of root) gives bett

er Mac App Store fidelity

installer: Warning: Command Line Tools (OS X 10.11).pkg was signed with a certif

icate that is not valid for store submission.

installer: Installation Check: Passed

installer: Volume Check: Passed

installer: Bundle com.apple.LLDB.framework will be installed to /Library/Develop

er/CommandLineTools/Library/PrivateFrameworks/LLDB.framework

installer: Bundle com.MyCompany.InstallerPlugin.TestPackage will be installed to

/System/Library/Frameworks/Python.framework/Examples/PyObjC/2.5.1/InstallerPlug

inSample/InstallerPluginsTestPackage.pkg

installer: Starting install

installer: Install 0.0% complete

installer: Install 5.1% complete

installer: Install 11.0% complete

installer: Install 15.1% complete

installer: Install 21.0% complete

installer: Install 25.1% complete

installer: Install 30.4% complete

installer: Install 35.1% complete

installer: Install 42.2% complete

installer: Install 86.9% complete

installer: Install 95.2% complete

installer: Install 100.0% complete

installer: Finished install

Is this complete?

I rebooted the VM instance since my previous post and I modified file a.c modified as follows…

#include "stdio.h"

int main ()
{
    printf("hello world\n");
}

…and this nicely outputs “hello world”. Then…

tst$ file a.out
a.out: Mach-O 64-bit executable x86_64
tst$

tst$ lldb a.out
(lldb) target create "a.out"
Current executable set to 'a.out' (x86_64).
(lldb) run
error: process exited with status -1 (unable to attach)
(lldb) run
error: process exited with status -1 (unable to attach)
(lldb) run
error: process exited with status -1 (unable to attach)
(lldb)

Running ps –ef from another session:

tst2$ ps -ef|grep lldb
  502 3015 908 0 10:24AM ttys000 0:00.22 /Library/Developer/CommandLineT
ools/usr/bin/lldb a.out

tst2$ ps -ef|grep 3015
  502 3015 908 0 10:24AM ttys000 0:00.22 /Library/Developer/CommandLineT
ools/usr/bin/lldb a.out
  502 3165 3015 0 10:25AM ttys000 0:00.00 /Users/tst/a.out
  502 3705 3015 0 10:27AM ttys000 0:00.00 /Users/tst/a.out
  502 3724 3015 0 10:27AM ttys000 0:00.00 /Users/tst/a.out
tst2$

tst2$ ps -ef|grep debugserver
tst2$

There doesn’t seem to be a debugserver. Am I missing that?

Exiting lldb:

(lldb) exit
tst$

New processes snapshot:

tst2$ ps -ef|grep 3015
tst2$

tst2$ ps -ef|grep a.out
  502 3165 1 0 10:25AM ttys000 0:00.00 /Users/tst/a.out
  502 3705 1 0 10:27AM ttys000 0:00.00 /Users/tst/a.out
  502 3724 1 0 10:27AM ttys000 0:00.00 /Users/tst/a.out
tst2$

Sudo kill -9 has no effect:

tst2$ sudo kill -9 2173
Password:
tst2$
tst2$ ps -ef|grep a.out
  502 3165 1 0 10:25AM ttys000 0:00.00 /Users/tst/a.out
  502 3705 1 0 10:27AM ttys000 0:00.00 /Users/tst/a.out
  502 3724 1 0 10:27AM ttys000 0:00.00 /Users/tst/a.out
tst2$

To have lldb use the official Xcode version of lldb's debugserver (assuming you have Xcode installed and aren't trying to use just the command line tools), you should be able to build with a command line like this:

xcodebuild -scheme desktop -configuration Debug DEBUGSERVER_USE_FROM_SYSTEM=1 <

tst$ xcodebuild
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer dire
ctory '/Library/Developer/CommandLineTools' is a command line tools instance
tst $

tst$ xcodebuild -scheme desktop -configuration Debug DEBUGSER
VER_USE_FROM_SYSTEM=1
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer dire
ctory '/Library/Developer/CommandLineTools' is a command line tools instance
tst$

Or from within Xcode itself, locally adjust your Xcode project to set the "DEBUGSERVER_USE_FROM_SYSTEM" user variable to 1. <

Not sure what “from within Xcode itself” means (GUI?) but I tried this:

tst$ export DEBUGSERVER_USE_FROM_SYSTEM=1
tst$ set|grep DEBUG
DEBUGSERVER_USE_FROM_SYSTEM=1

This doesn’t seem to help.

I'm not sure if you already did this, but you may need to turn on your dev tools security via:

sudo DevToolSecurity –enable <

tst$ sudo find / -name DevToolSecurity
find: /dev/fd/3: Not a directory
find: /dev/fd/4: Not a directory
find: /Volumes/VMware Shared Folders: Input/output error
tst$

I have a feeling that my install is not complete. What can I do from within the ssh session? As I’m remote (ssh only) I’d have to ask the local admin to tweak settings using the Xcode gui, if needed. What would you suggest?

Thanks,
Andre

Try a 'killall -CONT a.out' - your processes are paused, waiting for a debugger to attach, and they're probably not receiving signals.

The Command_Line_Tools package is a stripped down set of binaries intended for folks who need a compiler to build software.

The first time you run a debugger on a system (or possibly it's a login-session, I forget), a window is popped up asking you to verify that you want to authorize this. That doesn't work for people who are remotely logged in, so the DevToolSecurity binary that Todd mentioned exists. But it's probably not included in the Command_Line_Tools package. If you want to use a debugger, you'll either need GUI console access to your VM install, or you'll need to install the full xcode package (typically downloaded from the Mac App Store, but it may be on developer.apple.com in a .dmg that can be downloaded). xcodebuild is also not included in the Command_Line_Tools package, so you won't be able to rebuild lldb from source using that method.

J

If you ssh in you must enable developer mode one time per boot:

sudo /usr/sbin/DevToolsSecurity --enable

Then you should be able to debug. If you don't, it will popup a dialog box on the remote system that attempts to get authorization, but you will never see that in your SSH window...

Let me know if this helps.

Greg

Greg, I owe you a Bud (or fruit juice if you prefer) !!

That's all I had to do... Now it works like a charm!

tst$ sudo /usr/sbin/DevToolsSecurity --enable
Developer mode is now enabled.
tst$ lldb testabc
(lldb) target create "testabc"
Current executable set to 'testabc' (x86_64).
(lldb) run
Process 34322 launched: '/Users/tst/testabc' (x86_64)
TESTING on 2016/01/05 09:53:25
...

And all lldb commands work nicely.
Now disabling for a test:

tst$ sudo /usr/sbin/DevToolsSecurity --disable
Developer mode is now disabled.
tst$ lldb testabc
(lldb) target create "testabc"
Current executable set to 'testabc' (x86_64).
(lldb) run
error: process exited with status -1 (unable to attach)
(lldb) quit

So that was definitely the (only) reason.

Perhaps this 'developer mode' is something natural in the eyes of Mac developers. I do porting on multiple systems, I'm becoming more and more familiar with Mac every day, and I was not fully aware of the switch, the more that lldb ran 'almost' normally.

Wouldn't it be an idea to add to lldb a warning against missing developer mode rather than exhibiting its odd 'unable to attach' and zombie creation behavior? Even as 'developer' I still was not able to kill the zombie I created per the above 'developer disabled' lldb session.
So how about:

(lldb) run
error: developer mode not enabled

Thanks again!
Andre

Greg, I owe you a Bud (or fruit juice if you prefer) !!

That's all I had to do... Now it works like a charm!

tst$ sudo /usr/sbin/DevToolsSecurity --enable
Developer mode is now enabled.
tst$ lldb testabc
(lldb) target create "testabc"
Current executable set to 'testabc' (x86_64).
(lldb) run
Process 34322 launched: '/Users/tst/testabc' (x86_64)
TESTING on 2016/01/05 09:53:25
...

And all lldb commands work nicely.
Now disabling for a test:

tst$ sudo /usr/sbin/DevToolsSecurity --disable
Developer mode is now disabled.
tst$ lldb testabc
(lldb) target create "testabc"
Current executable set to 'testabc' (x86_64).
(lldb) run
error: process exited with status -1 (unable to attach)
(lldb) quit

So that was definitely the (only) reason.

Great!

Perhaps this 'developer mode' is something natural in the eyes of Mac developers. I do porting on multiple systems, I'm becoming more and more familiar with Mac every day, and I was not fully aware of the switch, the more that lldb ran 'almost' normally.

It is for security reasons that is has been added.

Wouldn't it be an idea to add to lldb a warning against missing developer mode rather than exhibiting its odd 'unable to attach' and zombie creation behavior? Even as 'developer' I still was not able to kill the zombie I created per the above 'developer disabled' lldb session.
So how about:

(lldb) run
error: developer mode not enabled

We should be able to do this. The main issue is detecting that the user is in a remote scenario where they don't have access to the UI. A dialog box will be popped up if the user is on the system, but when remotely connected we would need to detect this and return a correct error. This is a little harder as well because "debugserver", our GDB remote protocol debug stub, is what is requesting the debugging privelege. This is a program that is spawned by LLDB as a child process. But is should be able to be done somehow.

Greg

I have a low priority Todo to implement this - when I looked into it, it was just a few CF calls and I could retrieve whether developer mode was enabled in debugserver and report that to the user.