incorrect shared library addresses

I’m having a strange issue debugging a go program.

I set a breakpoint, and lldb stops at it. However, lldb thinks it’s in a different function:

(lldb) b wikilinktester.go:35
Breakpoint 1: where = wikilinktester`main.main + 805 at wikilinktester.go:35, address = 0x0000000000002365
(lldb) r
Process 43141 launched: ‘/tmp/gopath/bin/wikilinktester’ (x86_64)
Process 43141 stopped

  • thread #5: tid = 0x0001, 0x0000000000002365 libOpenScriptingUtil.dylib, stop reason = breakpoint 1.1
    frame #0: 0x0000000000002365 libOpenScriptingUtil.dylib
    → 0x2365: movq %rax, 0x48(%rsp)
    0x236a: movq %rax, (%rsp)
    0x236e: callq 0xb23d0
    0x2373: movq 0x8(%rsp), %rbx

For some reason lldb seems to think all the shared libraries are loaded at address 0:

(lldb) image list
[ 0] 0x0000000000001000 /tmp/gopath/bin/wikilinktester
[ 1] B1B370A5-479F-3533-8AD7-97B687D4F989 0x00007fff5fc00000 /usr/lib/dyld
[ 2] 1866C519-C5F3-3D09-8C17-A8F703664521 0x0000000000000000 /usr/lib/libSystem.B.dylib
[ 3] 5C0892B8-9691-341F-9279-CA3A74D59AA0 0x0000000000000000 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
[ 4] FCCCC4FD-043A-30CA-9997-4211CA0E9297 0x0000000000000000 /System/Library/Frameworks/Security.framework/Versions/A/Security
[ 5] 45E9A2E7-99C4-36B2-BEE3-0C4E11614AD1 0x0000000000000000 /usr/lib/system/libcache.dylib
[ 6] E789748D-F9A7-3CFF-B317-90DF348B1E95 0x0000000000000000 /usr/lib/system/libcommonCrypto.dylib
[ 7] BF8FC133-EE10-3DA6-9B90-92039E28678F 0x0000000000000000 /usr/lib/system/libcompiler_rt.dylib
[ 8] 0C68D3A6-ACDD-3EF3-991A-CC82C32AB836 0x0000000000000000 /usr/lib/system/libcopyfile.dylib
[ 9] 5779FFA0-4D9A-3AD4-B7F2-618227621DC8 0x0000000000000000 /usr/lib/system/libcorecrypto.dylib
[ 10] 502CF32B-669B-3709-8862-08188225E4F0 0x0000000000000000 /usr/lib/system/libdispatch.dylib
[ 11] CFBBE540-D503-3AFC-B5D6-644F1E69949B 0x0000000000000000 /usr/lib/system/libdyld.dylib
[ 12] 77845842-DE70-3CC5-BD01-C3D14227CED5 0x0000000000000000 /usr/lib/system/libkeymgr.dylib
[ 13] 4F81CA3A-D2CE-3030-A89D-42F3DAD7BA8F 0x0000000000000000 /usr/lib/system/liblaunch.dylib
[ 14] 126CA2ED-DE91-308F-8881-B9DAEC3C63B6 0x0000000000000000 /usr/lib/system/libmacho.dylib
[ 15] 7AF90041-2768-378A-925A-D83161863642 0x0000000000000000 /usr/lib/system/libquarantine.dylib
[ 16] 3485B5F4-6CE8-3C62-8DFD-8736ED6E8531 0x0000000000000000 /usr/lib/system/libremovefile.dylib
[ 17] F153AC5B-0542-356E-88C8-20A62CA704E2 0x0000000000000000 /usr/lib/system/libsystem_asl.dylib
[ 18] 9615D10A-FCA7-3BE4-AA1A-1B195DACE1A1 0x0000000000000000 /usr/lib/system/libsystem_blocks.dylib
[ 19] F0635E0F-FE4B-34DB-ACF9-A58C1E9070E9 0x0000000000000000 /usr/lib/system/libsystem_c.dylib
[ 20] 56F94DCE-DBDE-3615-8F07-DE6270D9F8BE 0x0000000000000000 /usr/lib/system/libsystem_configuration.dylib
[ 21] 41B7C578-5A53-31C8-A96F-C73E030B0938 0x0000000000000000 /usr/lib/system/libsystem_coreservices.dylib
[ 22] 155DA0A9-2046-332E-BFA3-D7974A51F731 0x0000000000000000 /usr/lib/system/libsystem_coretls.dylib
[ 23] 0CEB5910-843F-315C-A1DE-5D955A48A045 0x0000000000000000 /usr/lib/system/libsystem_dnssd.dylib
[ 24] 2E16C4B3-A327-3957-9C41-143911979A1E 0x0000000000000000 /usr/lib/system/libsystem_info.dylib
[ 25] 16AD15EF-3DAE-3F63-9D26-26CCE1920762 0x0000000000000000 /usr/lib/system/libsystem_kernel.dylib
[ 26] 1E12AB45-6D96-36D0-A226-F24D9FB0D9D6 0x0000000000000000 /usr/lib/system/libsystem_m.dylib
[ 27] DDA8928B-CC0D-3255-BD8A-3FEA0982B890 0x0000000000000000 /usr/lib/system/libsystem_malloc.dylib
[ 28] 6105C134-6722-3C0A-A4CE-5E1261E2E1CC 0x0000000000000000 /usr/lib/system/libsystem_network.dylib
[ 29] BA58B30B-8377-3B0A-8AE3-4F84021D9D4E 0x0000000000000000 /usr/lib/system/libsystem_networkextension.dylib
[ 30] 61147800-F320-3DAA-850C-BADF33855F29 0x0000000000000000 /usr/lib/system/libsystem_notify.dylib
[ 31] 64E34079-D712-3D66-9CE2-418624A5C040 0x0000000000000000 /usr/lib/system/libsystem_platform.dylib
[ 32] ACE90967-ECD0-3251-AEEB-461E3C6414F7 0x0000000000000000 /usr/lib/system/libsystem_pthread.dylib
[ 33] 3F5E973F-C702-31AC-97BC-05F5C195683C 0x0000000000000000 /usr/lib/system/libsystem_sandbox.dylib
[ 34] 581DAD0F-6B63-3A48-B63B-917AF799ABAA 0x0000000000000000 /usr/lib/system/libsystem_secinit.dylib
[ 35] D0E96837-3CF6-323D-B711-6DF6F660E530 0x0000000000000000 /usr/lib/system/libsystem_stats.dylib
[ 36] 840F5301-B55A-3078-90B9-FEFFD6CD741A 0x0000000000000000 /usr/lib/system/libsystem_trace.dylib
[ 37] 5676F7EA-C1DF-329F-B006-D2C3022B7D70 0x0000000000000000 /usr/lib/system/libunc.dylib
[ 38] BE7E51A0-B6EA-3A54-9CCA-9D88F683A6D6 0x0000000000000000 /usr/lib/system/libunwind.dylib
[ 39] 5C829202-962E-3744-8B50-00D38CC88E84 0x0000000000000000 /usr/lib/system/libxpc.dylib
[ 40] A260789B-D4D8-316A-9490-254767B8A5F1 0x0000000000000000 /usr/lib/libauto.dylib
[ 41] 2EE8E436-5CDC-34C5-9959-5BA218D507FB 0x0000000000000000 /usr/lib/libDiagnosticMessagesClient.dylib
[ 42] 3CD34752-B1F9-31D2-865D-B5B0F0BE3111 0x0000000000000000 /usr/lib/libicucore.A.dylib
[ 43] 759E155D-BC42-3D4E-869B-6F57D477177C 0x0000000000000000 /usr/lib/libobjc.A.dylib
[ 44] 88C7C7DE-04B8-316F-8B74-ACD9F3DE1AA1 0x0000000000000000 /usr/lib/libz.1.dylib
[ 45] A3A2E56C-2B65-37C7-B43A-A1F926E1A0BB 0x0000000000000000 /usr/lib/libbsm.0.dylib
[ 46] 7CD69BB5-97BA-3858-8A8B-2F33F129E6E7 0x0000000000000000 /usr/lib/libxar.1.dylib
[ 47] BCB1F77E-2484-3EC4-A1D5-1AE90A407A23 0x0000000000000000 /usr/lib/libsqlite3.dylib
[ 48] E805398D-9A92-31F8-8005-8DC188BD8B6E 0x0000000000000000 /usr/lib/libpam.2.dylib
[ 49] 09C0518C-90DF-3FC3-96D6-34D35F72C8EF 0x0000000000000000 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
[ 50] D6A2216D-ADB2-3F24-AD30-F6D00829F545 0x0000000000000000 /usr/lib/libOpenScriptingUtil.dylib
[ 51] 1B9530FD-989B-3174-BB1C-BDC159501710 0x0000000000000000 /usr/lib/libc++.1.dylib
[ 52] 88A22A0F-87C6-3002-BFBA-AC0F2808B8B9 0x0000000000000000 /usr/lib/libc++abi.dylib
[ 53] 2DF83FBC-5C08-39E1-94F5-C28653791B5F 0x0000000000000000 /usr/lib/libbz2.1.0.dylib
[ 54] 3FBA890F-2850-3A45-87EA-DB6892BDEB60 0x0000000000000000 /usr/lib/libxml2.2.dylib
[ 55] 2ADAE067-78A0-371E-A5A8-1E7C892D193C 0x0000000000000000 /usr/lib/system/libkxld.dylib

Any idea what’s going on here?

Breakpoint 1: where = wikilinktester`main.main + 805 at wikilinktester.go:35, address = 0x0000000000002365

The address of the breakpoint is kinda interesting - that seems suspiciously low for an OS X user executable.

Which lldb are you using? If you built it, what process did you use to build it, and which Xcode do you have?

I have the same behavior with the system lldb and the one I build in xcode.
My xcode is Version 6.4 (6E35b)
I just hit command-b to build, is that what you mean? I don’t really know xcode very well.

I am guessing that your binary is missing a LC_VERSION_MIN_XXX load command and so it is being loaded with "<arch>-unknown-unknown" as the architecture. This is probably causing LLDB to load the DynamicLoaderStatic as the dynamic loader which just loads all shared libraries as the address that is contained within the file itself, which is zero for most shared libraries.

If you have a linker that is generating mach-o files, you will need to add a LC_VERSION_MIN_MACOSX load command to your executable so we know it is MacOSX.

What does the output of "target list" show as the triple? It is probably "x86_64-unknown-unknown". We are often able to extract the correct triple for an executable from the executable itself. When no architecture if specified, it will default to the architecture that we detect in the main executable and this architecture will be used when a process asks for plug-ins to be created, line a DynamicLoader plug-in. If the triple is wrong, then we might select the wrong plug-ins for what you are trying to debug.

You might be able to get around this for now by specifying this when you create your target:

(lldb) target create --arch=x86_64-apple-macosx /tmp/gopath/bin/wikilinktester

Yep, it looks like LC_VERSION_MIN_MACOSX is missing.
target list shows:

target #0: /private/tmp/gopath/bin/wikilinktester ( arch=x86_64-apple-, platform=host )

specifying --arch doesn’t seem to work. It still has the modules loaded starting at 0, and target list says:

  • target #1: /tmp/gopath/bin/wikilinktester ( arch=x86_64h-apple-macosx, platform=host, pid=48597, state=stopped )

If you link a go program with c code it uses the system linker, which does add LC_VERSION_MIN_MACOSX. That explains why I’ve never seen this before. Here’s what that sort of binary looks like:

(lldb) target list
Current targets:

  • target #0: /private/tmp/gopath/bin/wikilinktester ( arch=x86_64-apple-macosx, platform=host, pid=49564, state=stopped )
    (lldb) image list
    [ 0] DC92B610-E1DD-3CB2-AD7E-4EEF21E19D20 0x0000000004000000 /private/tmp/gopath/bin/wikilinktester
    [ 1] B1B370A5-479F-3533-8AD7-97B687D4F989 0x00007fff5fc00000 /usr/lib/dyld
    [ 2] 1866C519-C5F3-3D09-8C17-A8F703664521 0x00007fff86ce6000 /usr/lib/libSystem.B.dylib
    [ 3] 5C0892B8-9691-341F-9279-CA3A74D59AA0 0x00007fff8631a000 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation

Glad we identified your problem. Can you guys always use the system linker? Any reason to use another linker? If you continue to a custom linker you will want to modify it to set these. If you compile Go programs that run on iOS simulator or iOS you will want to emit a LC_MIN_VERSION_IOS.

Greg Clayton

Well the go linker is actually what generates code from the IR, so it’s always required. And as it stands now you can cross compile from another platform as long as you don’t link with c code. I don’t know if they’d want to change that by always using the os x linker. I filed a bug for them to add the min version.

On a semi-related note, I’ve just discovered that the go compiler is writing line tables that increment the line number one instruction early. Would it make sense to try to work around this in LLDB?

Well the go linker is actually what generates code from the IR, so it's always required. And as it stands now you can cross compile from another platform as long as you don't link with c code. I don't know if they'd want to change that by always using the os x linker. I filed a bug for them to add the min version.

Sounds good.

On a semi-related note, I've just discovered that the go compiler is writing line tables that increment the line number one instruction early. Would it make sense to try to work around this in LLDB?

We should fix the compiler for this issue.