Hi,
I’m having a strange issue on Linux regarding LLDB not recognising a couple of the shared libraries that my C/C++ application is loading. This first became apparent when I tried to place breakpoints in these libraries when the application was running, and VS Code reported that there were no locations available. I’ve subsequently run image list in the LLDB console, and although I can see most of the libraries my application is using, the two specific ones I need to place breakpoints in are not present. The libraries are definitely being used by the application, because otherwise it would not be able to function.
I’ve tried rebuilding, re-cloning, going back in Git history and even reinstalling my system (I needed to anyway), but none of this helps. Is there any way I can diagnose what’s going wrong here?
Are the libraries loaded automatically or manually with dlopen()?
If you run ldd (program that lists an application’s shared library dependencies and resolutions) on your application, are the libraries listed?
What does the file program output for the libraries?
The libraries concerned are loaded using dlopen() with the RTLD_NOW flag. As such, ldd lists:
linux-vdso.so.1 (0x00007fffaa4de000)
libfilesystem_stdio.so => ./libfilesystem_stdio.so (0x0000785f70830000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000774c0c400000)
/lib64/ld-linux-x86-64.so.2 (0x0000774c0c816000)
I think these are the only ones specified via the build configuration.
file result for the executable and the libraries that are causing issues:
$ file bond
bond: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=fafe6604ccc4a030157c5e1bdb5516d852bd0049, for GNU/Linux 3.2.0, with debug_info, not stripped
$ file nfopen/cl_dlls/client_amd64.so
nfopen/cl_dlls/client_amd64.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=789b573dcd6e652f2faf60c8554dea8546bb35a1, with debug_info, not stripped
$ file nfopen/dlls/server_amd64.so
nfopen/dlls/server_amd64.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=a10578517af3b791b3264115e7099ea86ab9e4a1, with debug_info, not stripped
You can check the dynamic loader logs:
(lldb) run
lldb DynamicLoaderDarwin::UseDYLDSPI: Use old DynamicLoader plugin
lldb DynamicLoaderDarwin::UseDYLDSPI: Use old DynamicLoader plugin
lldb DYLDRendezvous::UpdateExecutablePath exe module executable path set: '/tmp/test.o'
lldb DynamicLoaderPOSIXDYLD::DidLaunch()
lldb DynamicLoaderPOSIXDYLD::DidLaunch about to call ProbeEntry()
lldb Rendezvous structure is not set up yet. Trying to locate rendezvous breakpoint in the interpreter by symbol name.
lldb Successfully set rendezvous breakpoint at address 0xfffff7fdae90 for pid 3664655
When it runs lldb should be finding a place to hook into it.
Then there should be some events when the load happens:
(lldb) image list
[ 0] 73555B38-5F88-1AE1-B41B-30FF8729B36D-9FED880A 0x0000aaaaaaaaa000 /tmp/test.o
[ 1] 5594C839-A644-4A80-1E76-10B1BA8DE089-6218ED3C 0x0000fffff7fcc000 /usr/lib/aarch64-linux-gnu/ld-2.31.so
/usr/lib/debug/.build-id/55/94c839a6444a801e7610b1ba8de0896218ed3c.debug
[ 2] 8A3F013D-F292-A2BC-FE56-490D2244E9A8-A6162A47 0x0000fffff7ffc000 [vdso] (0x0000fffff7ffc000)
[ 3] CA7E577B-9F25-4594-8464-2C72D6090750-88330F7C 0x0000fffff7fb8000 /lib/aarch64-linux-gnu/libdl.so.2
/usr/lib/debug/.build-id/ca/7e577b9f25459484642c72d609075088330f7c.debug
[ 4] 127E5908-798B-8D0C-CC04-F49933AF3BEF-807F9E65 0x0000fffff7e45000 /lib/aarch64-linux-gnu/libc.so.6
/usr/lib/debug/.build-id/12/7e5908798b8d0ccc04f49933af3bef807f9e65.debug
(lldb) log enable lldb dyld
(lldb) process status
Process 3664546 stopped
* thread #1, name = 'test.o', stop reason = breakpoint 1.1
frame #0: 0x0000aaaaaaaaa794 test.o`main at test.c:7:18
4 #include <dlfcn.h>
5
6 int main() {
-> 7 void *handle = dlopen("/tmp/shared.so", RTLD_NOW);
8 return 0;
9 }
(lldb) n
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 3664546
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0xfffff7fff1d8
intern-state m_previous Rendezvous: version = 1, map_addr = 0x0000fffff7fff210, brk = 0x0000fffff7fdae90, state = 0 (eConsistent), ldbase = 0x0000fffff7fcc000
intern-state m_current Rendezvous: version = 1, map_addr = 0x0000fffff7fff210, brk = 0x0000fffff7fdae90, state = 1 (eAdd), ldbase = 0x0000fffff7fcc000
intern-state bool DYLDRendezvous::UpdateSOEntriesFromRemote() action = eNoAction
intern-state bool DYLDRendezvous::UpdateSOEntries() action = eNoAction
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 3664546 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 3664546
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0xfffff7fff1d8
intern-state m_previous Rendezvous: version = 1, map_addr = 0x0000fffff7fff210, brk = 0x0000fffff7fdae90, state = 1 (eAdd), ldbase = 0x0000fffff7fcc000
intern-state m_current Rendezvous: version = 1, map_addr = 0x0000fffff7fff210, brk = 0x0000fffff7fdae90, state = 0 (eConsistent), ldbase = 0x0000fffff7fcc000
intern-state bool DYLDRendezvous::UpdateSOEntriesFromRemote() action = eAddModules
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 3664546 stop_when_images_change=false
Process 3664546 stopped
* thread #1, name = 'test.o', stop reason = step over
frame #0: 0x0000aaaaaaaaa7a8 test.o`main at test.c:8:10
5
6 int main() {
7 void *handle = dlopen("/tmp/shared.so", RTLD_NOW);
-> 8 return 0;
9 }
(lldb) image list
< same as before plus... >
[ 5] 2A454363-8375-41E8-91B4-0DE41E62E56F-A27CCD40 0x0000fffff7e31000 /tmp/shared.so
Sorry for the delay in replying, I’ve been pretty busy lately. Following your steps, I get these logs:
Console is in 'commands' mode, prefix expressions with '?'.
Launching: /home/vesper/Documents/nightfire-open/build/install/nightfire-open/bond -dev 5 -windowed -width 1920 -height 1080
Launched process 9713
image list
[ 0] FAFE6604-CCC4-A030-157C-5E1BDB5516D8-52BD0049 0x0000555555554000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/bond
[ 1] 78FFE99E-D77A-37AE-9416-F1530F911018-8897CF80 0x00007ffff7fc6000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/usr/lib/debug/.build-id/78/ffe99ed77a37ae9416f1530f9110188897cf80.debug
[ 2] 7C7C346B-0A97-86E0-F933-1FE93D2E0853-D7EC209A 0x00007ffff7fc4000 [vdso] (0x00007ffff7fc4000)
[ 3] F8B581F4-00FD-CA58-5A46-95A136E4F11C-5580E739 0x00007ffff7f93000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libfilesystem_stdio.so
[ 4] 6A981B07-A373-1293-C24C-10A21397416D-3C3D52ED 0x00007ffff7c00000 /lib/x86_64-linux-gnu/libc.so.6
/usr/lib/debug/.build-id/6a/981b07a3731293c24c10a21397416d3c3d52ed.debug
[ 5] 7E45707E-4003-8B18-C6E2-4EB3F25AE3C9-D4E545FB 0x00007ffff6800000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libengine.so
[ 6] 3093B51E-294D-68E6-BF92-4F95E5DA5D84-4FB1C2A0 0x00007ffff4c00000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libref_gl.so
[ 7] 29CB054A-B7C4-D5D7-73D8-D75905971D41-6D2B8033 0x00007ffff4a2e000 /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
[ 8] 2D81A912-E7C8-5129-E41E-4C65B6C5BB81-80977A17 0x00007ffff7e84000 /lib/x86_64-linux-gnu/libm.so.6
/usr/lib/debug/.build-id/2d/81a912e7c85129e41e4c65b6c5bb8180977a17.debug
[ 9] EB89866A-3353-1C26-A57A-E843CCC284EB-9E1E862C 0x00007ffff7afa000 /lib/x86_64-linux-gnu/libasound.so.2
[ 10] 2053D833-337A-8B48-9611-18E9216273BB-EAAC0D5A 0x00007ffff7e2f000 /lib/x86_64-linux-gnu/libpulse.so.0
[ 11] 1B513FEF-E6DA-5E4C-F5B9-49B5953B2607-4B8F3508 0x00007ffff6692000 /lib/x86_64-linux-gnu/libsamplerate.so.0
[ 12] 788CA490-7059-CCAE-3B22-2E91F57038D2-5F65C55A 0x00007ffff48f0000 /lib/x86_64-linux-gnu/libX11.so.6
[ 13] 9FB1880E-02DF-A11A-8C39-CD1A170109DE-08302059 0x00007ffff7e1a000 /lib/x86_64-linux-gnu/libXext.so.6
[ 14] B2AF141F-7AC4-105C-8EA1-FA86FEC7BD84-A0DA34DC 0x00007ffff7e0e000 /lib/x86_64-linux-gnu/libXcursor.so.1
[ 15] 8FF5A3AC-871A-90FD-9D0A-7917C61F748A-41C6B5EE 0x00007ffff7ae6000 /lib/x86_64-linux-gnu/libXi.so.6
[ 16] E17BCEE8-CBB6-88A8-BA47-55637363836F-69FCA712 0x00007ffff7ade000 /lib/x86_64-linux-gnu/libXfixes.so.3
[ 17] B566CC30-96F7-8D3A-D546-9B0AC5A77BCB-FCD0306B 0x00007ffff7ad1000 /lib/x86_64-linux-gnu/libXrandr.so.2
[ 18] 0C9B5623-0C8F-1E18-E35C-40FC8D69756F-DF98424F 0x00007ffff7acc000 /lib/x86_64-linux-gnu/libXss.so.1
[ 19] C527CB73-E9CC-745A-5D9F-224835F1A6B6-F1FEC6A7 0x00007ffff667b000 /lib/x86_64-linux-gnu/libdrm.so.2
[ 20] 94094846-3CE4-3344-A436-F32C5910E19A-D1742AF9 0x00007ffff666a000 /lib/x86_64-linux-gnu/libgbm.so.1
[ 21] 038C2CA2-5696-D431-497B-1D32F511C40D-BA53D869 0x00007ffff7ac7000 /lib/x86_64-linux-gnu/libwayland-egl.so.1
[ 22] 079D2158-D05A-BF39-5004-2B2EA323C77A-AA425F66 0x00007ffff6659000 /lib/x86_64-linux-gnu/libwayland-client.so.0
[ 23] 7D81A21D-B487-F250-446C-263CFF38DE9B-4BCF25F5 0x00007ffff664f000 /lib/x86_64-linux-gnu/libwayland-cursor.so.0
[ 24] 99224C42-BDCE-B4FF-6A67-7C37F300B334-197748B8 0x00007ffff48aa000 /lib/x86_64-linux-gnu/libxkbcommon.so.0
[ 25] 1BA5A255-BD2F-22FF-4CBF-5F76CB5002D9-1C748F64 0x00007ffff489f000 /lib/x86_64-linux-gnu/libdecor-0.so.0
[ 26] 8A1D04C2-DB7B-6D18-1ECE-C036F7A39D03-B1CF14AA 0x00007ffff481b000 /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-16.1.so
[ 27] F7BF9927-5ACC-7423-5AD6-E20BEE2EEDB2-FEDF6EF4 0x00007ffff47cc000 /lib/x86_64-linux-gnu/libdbus-1.so.3
[ 28] CAB14959-65E4-F64E-0AF5-FAE4375614D2-DDCA189D 0x00007ffff47a2000 /lib/x86_64-linux-gnu/libxcb.so.1
[ 29] C3D60B68-9DEB-D95A-1A47-C5720BCC5270-0BD9BDE6 0x00007ffff4795000 /lib/x86_64-linux-gnu/libXrender.so.1
[ 30] F691D1B1-80F5-3BAE-80AF-27240EBF7A4A-ED01F8A4 0x00007ffff477f000 /lib/x86_64-linux-gnu/libwayland-server.so.0
[ 31] 985EC2DF-5352-AFC3-79A3-65934D71CDA9-673DC05E 0x00007ffff4754000 /lib/x86_64-linux-gnu/libexpat.so.1
[ 32] A26999B5-FAC5-344F-4035-700A43B9AF1E-50C490BA 0x00007ffff4742000 /lib/x86_64-linux-gnu/libxcb-randr.so.0
[ 33] 623A8441-7A0B-2D6C-FDDC-3F00E155A710-9C8A7000 0x00007ffff4737000 /lib/x86_64-linux-gnu/libffi.so.8
[ 34] A3FD8EC1-EEEF-319C-FC08-FFD57D948EFD-D472AD8F 0x00007ffff46ae000 /lib/x86_64-linux-gnu/libsndfile.so.1
[ 35] 364057BF-F6D8-F021-8030-924D7430DAA1-019D451B 0x00007ffff664a000 /lib/x86_64-linux-gnu/libX11-xcb.so.1
[ 36] 1380F118-DB9A-3032-9126-A922A7EBA22D-B344BFEE 0x00007ffff45d8000 /lib/x86_64-linux-gnu/libsystemd.so.0
[ 37] 425FF749-39B8-FD28-22B5-A137C68B4C73-37A04465 0x00007ffff45d0000 /lib/x86_64-linux-gnu/libasyncns.so.0
[ 38] 8D540FF2-7A88-13C4-F703-88EC40E7986E-87E640D6 0x00007ffff45bb000 /lib/x86_64-linux-gnu/libapparmor.so.1
[ 39] 7089B383-CACB-FC17-6063-4A3BE19A923E-51FE3315 0x00007ffff45b5000 /lib/x86_64-linux-gnu/libXau.so.6
[ 40] 6B60F995-04AA-1D39-99EA-02A14366D1A3-9D6C5DCF 0x00007ffff45ad000 /lib/x86_64-linux-gnu/libXdmcp.so.6
[ 41] AA1C8343-AC2E-166B-D710-FE94ECFC0A76-244C1F34 0x00007ffff454a000 /lib/x86_64-linux-gnu/libFLAC.so.12
[ 42] FC9C8326-76AA-C541-183D-1B4774746BEC-48B66051 0x00007ffff451d000 /lib/x86_64-linux-gnu/libvorbis.so.0
[ 43] 5D30E2A7-1366-9895-DCF3-B298FFED2E4C-E56121E8 0x00007ffff4472000 /lib/x86_64-linux-gnu/libvorbisenc.so.2
[ 44] A5F38B29-C9CC-7D49-20EF-B9624DAEBDEC-517EE117 0x00007ffff4413000 /lib/x86_64-linux-gnu/libopus.so.0
[ 45] 41451767-1DBF-2CEE-9060-06111E851679-AC4480B3 0x00007ffff4409000 /lib/x86_64-linux-gnu/libogg.so.0
[ 46] 60C84822-032A-4A7C-F279-CEBF21E23944-4ECB5EF2 0x00007ffff43ad000 /lib/x86_64-linux-gnu/libmpg123.so.0
[ 47] F4F49D23-05AA-7AAC-3007-9FB4EC0FD905-FD89207D 0x00007ffff4337000 /lib/x86_64-linux-gnu/libmp3lame.so.0
[ 48] B77AF3C5-336D-10F1-17AA-3A40068CFD66-7C36AE1E 0x00007ffff432b000 /lib/x86_64-linux-gnu/libcap.so.2
[ 49] 8E0096EC-9492-930B-1738-28A13711F0E8-E5E93C24 0x00007ffff41e3000 /lib/x86_64-linux-gnu/libgcrypt.so.20
[ 50] 5160D650-2517-3B93-E542-C0D6A362A0BA-4D79DFA1 0x00007ffff41b1000 /lib/x86_64-linux-gnu/liblzma.so.5
[ 51] E0B4FAC4-F802-5115-CA09-5EA829E89F16-756203BD 0x00007ffff40fa000 /lib/x86_64-linux-gnu/libzstd.so.1
[ 52] 0ECCA711-8D86-DC1D-D138-D3E8009F4D2B-87D1B1B1 0x00007ffff40d7000 /lib/x86_64-linux-gnu/liblz4.so.1
[ 53] 44DDA138-D768-A9E3-435D-4821FD823366-1CFB8AEE 0x00007ffff40c2000 /lib/x86_64-linux-gnu/libbsd.so.0
[ 54] 04D79670-126B-0A62-8C85-794C5166395A-F6CC62E9 0x00007ffff409c000 /lib/x86_64-linux-gnu/libgpg-error.so.0
[ 55] 1D6B1FA7-7179-C1C2-C7E4-09746DFF3620-A84CB0F8 0x00007ffff408d000 /lib/x86_64-linux-gnu/libmd.so.0
[ 56] DC4A87CA-42A7-464D-A98A-B3FE3703AF68-CD7D38F1 0x00007ffff340c000 /lib/x86_64-linux-gnu/libudev.so.1
log enable lldb dyld
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 9713
intern-state DYLDRendezvous::Resolve address size: 8, padding 4
intern-state DYLDRendezvous::Resolve cursor = 0x7ffff7ffe108
intern-state DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 9713 stop_when_images_change=false
Stop reason: signal SIGSTOP
image list
[ 0] FAFE6604-CCC4-A030-157C-5E1BDB5516D8-52BD0049 0x0000555555554000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/bond
[ 1] 78FFE99E-D77A-37AE-9416-F1530F911018-8897CF80 0x00007ffff7fc6000 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/usr/lib/debug/.build-id/78/ffe99ed77a37ae9416f1530f9110188897cf80.debug
[ 2] 7C7C346B-0A97-86E0-F933-1FE93D2E0853-D7EC209A 0x00007ffff7fc4000 [vdso] (0x00007ffff7fc4000)
[ 3] F8B581F4-00FD-CA58-5A46-95A136E4F11C-5580E739 0x00007ffff7f93000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libfilesystem_stdio.so
[ 4] 6A981B07-A373-1293-C24C-10A21397416D-3C3D52ED 0x00007ffff7c00000 /lib/x86_64-linux-gnu/libc.so.6
/usr/lib/debug/.build-id/6a/981b07a3731293c24c10a21397416d3c3d52ed.debug
[ 5] 7E45707E-4003-8B18-C6E2-4EB3F25AE3C9-D4E545FB 0x00007ffff6800000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libengine.so
[ 6] 3093B51E-294D-68E6-BF92-4F95E5DA5D84-4FB1C2A0 0x00007ffff4c00000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libref_gl.so
[ 7] 29CB054A-B7C4-D5D7-73D8-D75905971D41-6D2B8033 0x00007ffff4a2e000 /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
[ 8] 2D81A912-E7C8-5129-E41E-4C65B6C5BB81-80977A17 0x00007ffff7e84000 /lib/x86_64-linux-gnu/libm.so.6
/usr/lib/debug/.build-id/2d/81a912e7c85129e41e4c65b6c5bb8180977a17.debug
[ 9] EB89866A-3353-1C26-A57A-E843CCC284EB-9E1E862C 0x00007ffff7afa000 /lib/x86_64-linux-gnu/libasound.so.2
[ 10] 2053D833-337A-8B48-9611-18E9216273BB-EAAC0D5A 0x00007ffff7e2f000 /lib/x86_64-linux-gnu/libpulse.so.0
[ 11] 1B513FEF-E6DA-5E4C-F5B9-49B5953B2607-4B8F3508 0x00007ffff6692000 /lib/x86_64-linux-gnu/libsamplerate.so.0
[ 12] 788CA490-7059-CCAE-3B22-2E91F57038D2-5F65C55A 0x00007ffff48f0000 /lib/x86_64-linux-gnu/libX11.so.6
[ 13] 9FB1880E-02DF-A11A-8C39-CD1A170109DE-08302059 0x00007ffff7e1a000 /lib/x86_64-linux-gnu/libXext.so.6
[ 14] B2AF141F-7AC4-105C-8EA1-FA86FEC7BD84-A0DA34DC 0x00007ffff7e0e000 /lib/x86_64-linux-gnu/libXcursor.so.1
[ 15] 8FF5A3AC-871A-90FD-9D0A-7917C61F748A-41C6B5EE 0x00007ffff7ae6000 /lib/x86_64-linux-gnu/libXi.so.6
[ 16] E17BCEE8-CBB6-88A8-BA47-55637363836F-69FCA712 0x00007ffff7ade000 /lib/x86_64-linux-gnu/libXfixes.so.3
[ 17] B566CC30-96F7-8D3A-D546-9B0AC5A77BCB-FCD0306B 0x00007ffff7ad1000 /lib/x86_64-linux-gnu/libXrandr.so.2
[ 18] 0C9B5623-0C8F-1E18-E35C-40FC8D69756F-DF98424F 0x00007ffff7acc000 /lib/x86_64-linux-gnu/libXss.so.1
[ 19] C527CB73-E9CC-745A-5D9F-224835F1A6B6-F1FEC6A7 0x00007ffff667b000 /lib/x86_64-linux-gnu/libdrm.so.2
[ 20] 94094846-3CE4-3344-A436-F32C5910E19A-D1742AF9 0x00007ffff666a000 /lib/x86_64-linux-gnu/libgbm.so.1
[ 21] 038C2CA2-5696-D431-497B-1D32F511C40D-BA53D869 0x00007ffff7ac7000 /lib/x86_64-linux-gnu/libwayland-egl.so.1
[ 22] 079D2158-D05A-BF39-5004-2B2EA323C77A-AA425F66 0x00007ffff6659000 /lib/x86_64-linux-gnu/libwayland-client.so.0
[ 23] 7D81A21D-B487-F250-446C-263CFF38DE9B-4BCF25F5 0x00007ffff664f000 /lib/x86_64-linux-gnu/libwayland-cursor.so.0
[ 24] 99224C42-BDCE-B4FF-6A67-7C37F300B334-197748B8 0x00007ffff48aa000 /lib/x86_64-linux-gnu/libxkbcommon.so.0
[ 25] 1BA5A255-BD2F-22FF-4CBF-5F76CB5002D9-1C748F64 0x00007ffff489f000 /lib/x86_64-linux-gnu/libdecor-0.so.0
[ 26] 8A1D04C2-DB7B-6D18-1ECE-C036F7A39D03-B1CF14AA 0x00007ffff481b000 /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-16.1.so
[ 27] F7BF9927-5ACC-7423-5AD6-E20BEE2EEDB2-FEDF6EF4 0x00007ffff47cc000 /lib/x86_64-linux-gnu/libdbus-1.so.3
[ 28] CAB14959-65E4-F64E-0AF5-FAE4375614D2-DDCA189D 0x00007ffff47a2000 /lib/x86_64-linux-gnu/libxcb.so.1
[ 29] C3D60B68-9DEB-D95A-1A47-C5720BCC5270-0BD9BDE6 0x00007ffff4795000 /lib/x86_64-linux-gnu/libXrender.so.1
[ 30] F691D1B1-80F5-3BAE-80AF-27240EBF7A4A-ED01F8A4 0x00007ffff477f000 /lib/x86_64-linux-gnu/libwayland-server.so.0
[ 31] 985EC2DF-5352-AFC3-79A3-65934D71CDA9-673DC05E 0x00007ffff4754000 /lib/x86_64-linux-gnu/libexpat.so.1
[ 32] A26999B5-FAC5-344F-4035-700A43B9AF1E-50C490BA 0x00007ffff4742000 /lib/x86_64-linux-gnu/libxcb-randr.so.0
[ 33] 623A8441-7A0B-2D6C-FDDC-3F00E155A710-9C8A7000 0x00007ffff4737000 /lib/x86_64-linux-gnu/libffi.so.8
[ 34] A3FD8EC1-EEEF-319C-FC08-FFD57D948EFD-D472AD8F 0x00007ffff46ae000 /lib/x86_64-linux-gnu/libsndfile.so.1
[ 35] 364057BF-F6D8-F021-8030-924D7430DAA1-019D451B 0x00007ffff664a000 /lib/x86_64-linux-gnu/libX11-xcb.so.1
[ 36] 1380F118-DB9A-3032-9126-A922A7EBA22D-B344BFEE 0x00007ffff45d8000 /lib/x86_64-linux-gnu/libsystemd.so.0
[ 37] 425FF749-39B8-FD28-22B5-A137C68B4C73-37A04465 0x00007ffff45d0000 /lib/x86_64-linux-gnu/libasyncns.so.0
[ 38] 8D540FF2-7A88-13C4-F703-88EC40E7986E-87E640D6 0x00007ffff45bb000 /lib/x86_64-linux-gnu/libapparmor.so.1
[ 39] 7089B383-CACB-FC17-6063-4A3BE19A923E-51FE3315 0x00007ffff45b5000 /lib/x86_64-linux-gnu/libXau.so.6
[ 40] 6B60F995-04AA-1D39-99EA-02A14366D1A3-9D6C5DCF 0x00007ffff45ad000 /lib/x86_64-linux-gnu/libXdmcp.so.6
[ 41] AA1C8343-AC2E-166B-D710-FE94ECFC0A76-244C1F34 0x00007ffff454a000 /lib/x86_64-linux-gnu/libFLAC.so.12
[ 42] FC9C8326-76AA-C541-183D-1B4774746BEC-48B66051 0x00007ffff451d000 /lib/x86_64-linux-gnu/libvorbis.so.0
[ 43] 5D30E2A7-1366-9895-DCF3-B298FFED2E4C-E56121E8 0x00007ffff4472000 /lib/x86_64-linux-gnu/libvorbisenc.so.2
[ 44] A5F38B29-C9CC-7D49-20EF-B9624DAEBDEC-517EE117 0x00007ffff4413000 /lib/x86_64-linux-gnu/libopus.so.0
[ 45] 41451767-1DBF-2CEE-9060-06111E851679-AC4480B3 0x00007ffff4409000 /lib/x86_64-linux-gnu/libogg.so.0
[ 46] 60C84822-032A-4A7C-F279-CEBF21E23944-4ECB5EF2 0x00007ffff43ad000 /lib/x86_64-linux-gnu/libmpg123.so.0
[ 47] F4F49D23-05AA-7AAC-3007-9FB4EC0FD905-FD89207D 0x00007ffff4337000 /lib/x86_64-linux-gnu/libmp3lame.so.0
[ 48] B77AF3C5-336D-10F1-17AA-3A40068CFD66-7C36AE1E 0x00007ffff432b000 /lib/x86_64-linux-gnu/libcap.so.2
[ 49] 8E0096EC-9492-930B-1738-28A13711F0E8-E5E93C24 0x00007ffff41e3000 /lib/x86_64-linux-gnu/libgcrypt.so.20
[ 50] 5160D650-2517-3B93-E542-C0D6A362A0BA-4D79DFA1 0x00007ffff41b1000 /lib/x86_64-linux-gnu/liblzma.so.5
[ 51] E0B4FAC4-F802-5115-CA09-5EA829E89F16-756203BD 0x00007ffff40fa000 /lib/x86_64-linux-gnu/libzstd.so.1
[ 52] 0ECCA711-8D86-DC1D-D138-D3E8009F4D2B-87D1B1B1 0x00007ffff40d7000 /lib/x86_64-linux-gnu/liblz4.so.1
[ 53] 44DDA138-D768-A9E3-435D-4821FD823366-1CFB8AEE 0x00007ffff40c2000 /lib/x86_64-linux-gnu/libbsd.so.0
[ 54] 04D79670-126B-0A62-8C85-794C5166395A-F6CC62E9 0x00007ffff409c000 /lib/x86_64-linux-gnu/libgpg-error.so.0
[ 55] 1D6B1FA7-7179-C1C2-C7E4-09746DFF3620-A84CB0F8 0x00007ffff408d000 /lib/x86_64-linux-gnu/libmd.so.0
[ 56] DC4A87CA-42A7-464D-A98A-B3FE3703AF68-CD7D38F1 0x00007ffff340c000 /lib/x86_64-linux-gnu/libudev.so.1
[ 57] 10266177-5A59-C091-23F2-26BC3021556F-D42BC563 0x00007ffff0c00000 /lib/x86_64-linux-gnu/libstdc++.so.6
[ 58] 56A40DB5-8FDB-8A2A-B280-771A135196CE-D0285012 0x00007ffff337c000 /lib/x86_64-linux-gnu/libgcc_s.so.1
[ 59] 8B8825E4-B79D-1807-D3B6-A355DAC99731-681BDCEE 0x00007ffff0b79000 /lib/x86_64-linux-gnu/libGL.so.1
[ 60] 898843B1-B538-6677-5DA1-84183778F71C-1BA576AE 0x00007ffff0ac1000 /lib/x86_64-linux-gnu/libGLdispatch.so.0
[ 61] 75CE2F21-C19E-6516-E066-D7EB8D09E65D-438913FE 0x00007ffff3267000 /lib/x86_64-linux-gnu/libGLX.so.0
[ 62] FE7B0C1E-BF30-251D-936F-661E527B0EA3-A266ECEC 0x00007ffff099b000 /lib/x86_64-linux-gnu/libGLX_nvidia.so.0
[ 63] A698FCCF-79B9-85F6-5C74-48F181863A70-78F851C2 0x00007ffff0600000 /lib/x86_64-linux-gnu/libnvidia-glsi.so.525.147.05
[ 64] FD05A6A8-D51B-C006-E08D-A0DF2525DBC3-4C3FBA9F 0x00007ffff7f7c000 /lib/x86_64-linux-gnu/libnvidia-tls.so.525.147.05
[ 65] EFA82CF2-2E08-F320-DA19-2960FFA47756-33F0BC9E 0x00007fffeda00000 /lib/x86_64-linux-gnu/libnvidia-glcore.so.525.147.05
[ 66] 724425B7-B601-A555-2FAC-4E3392039927-1FD1A01D 0x00007ffff7f77000 /lib/x86_64-linux-gnu/libdl.so.2
/usr/lib/debug/.build-id/72/4425b7b601a5552fac4e33920399271fd1a01d.debug
[ 67] 6659DE08-3272-A1C8-8A67-2E06A37E7416-6431045C 0x00007ffff7f72000 /lib/x86_64-linux-gnu/libpthread.so.0
/usr/lib/debug/.build-id/66/59de083272a1c88a672e06a37e74166431045c.debug
[ 68] A9FF2C10-85AF-A129-85B5-BA279C63021A-23BBFC5F 0x00007ffff6645000 /lib/x86_64-linux-gnu/librt.so.1
/usr/lib/debug/.build-id/a9/ff2c1085afa12985b5ba279c63021a23bbfc5f.debug
[ 69] AE675DE2-711E-ECDE-6D84-6CBA36F5E436-F6DFD289 0x00007ffff33f0000 /lib/x86_64-linux-gnu/libxcb-glx.so.0
[ 70] 72F9A3E5-1052-F75C-D938-8920A0F213DC-E014830D 0x00007fffed72d000 /home/vesper/Documents/nightfire-open/build/install/nightfire-open/libmenu.so
The libraries that are having issues do not show up in the image list.
So your program “bond” is loading “client_amd64.so” and “server_amd64.so”, right?
How does it find them? I mean, what path gets passed to dlopen()? I wonder if the load is going through, but lldb can’t find the file. I’d still expect it to be in the image list, though.
Unfortunately, the dyld log for the POSIX DYLD plugin doesn’t print out the file and path it gets from the link map, so we don’t know where lldb is looking for the files. Are you running out of the directory with bond? Maybe we can figure out a path mapping to set up (see “help image search-paths add”).
It is worth saying that these .so files don’t reside in the LD_LIBRARY_PATH - this is an open-source Half Life engine reimplementation, so the engine just loads the libraries from the mod’s client and server directories. I can try modifying the LD_LIBRARY_PATH to include these directories and see if it makes any difference.
I can’t remember whether the paths are computed by the engine as relative or absolute, so I can also double-check this and see if going one or the other way helps.
Interesting. If I change the dlopen() call to use an absolute path to the library, the breakpoints do get loaded. I guess I could either work out a way to resolve the absolute path before loading the library, or change the LLDB config to be able to work out the relative path properly, if that’s something that can be achieved?
“image search-paths add” is the way to add a mapping to find shared libraries.
What is the path given to dlopen() to find a given library?
After further inspection, it seems the engine does passes a path relative to the executable directory - for example, the server library is passed as nfopen/dlls/server_amd64.so. This also happens to be relative to LD_LIBRARY_PATH, since that’s set to the executable directory by the application launch script.
image search-paths add looks like it’s used to substitute a new search path for an old one. How exactly should I be using this? I don’t want to replace a search path, just add a new one.
It’s a textual substitution, so “image search paths add nfopen /usr/local/foo/nfopen” will replace “nfopen/dlls/server_amd64.so” with “/usr/local/foo/nfopen/dlls/server_amd64.so”. Case sensitivity matters, so it won’t match NFOPEN with nfopen (or c:\ with C:\ for Windows).
This doesn’t seem to work for me. I’ve tried image search-paths add "nfopen/" "/home/vesper/Documents/nightfire-open/build/install/nightfire-open/nfopen/" to match the beginning of nfopen/dlls/server_amd64.so, but the library still doesn’t appear to be found.
Unfortunately, we don’t know the exact path that the dynamic loader is putting in the link map. That’s something that the dyld logging should print, but it doesn’t, so that doesn’t help. You’d have to look at the link map to see, but we’d need to know what global holds the link map (it varies from OS to OS).
What OS are you running on?
Step over a dlopen() call, then run “breakpoint list -i” to see the internal breakpoints. What does that output?