ld.lld drops relocations

Hi,

i’m trying to dig issue with building systemd-boot EFI images with llvm/clang and linking with llvm/lld. In general building and linking process works and i get my final objects. Unfortunately these EFI images does not boot system. I noticed that .efi image is missing PE/COFF relocations.
When i changed linker to ld.bfd, voila everything is fine and final efi image does boot system and have needed relocs. I tried all the ld.lld options related to relocations, but still no success.
Thanks for your help.

Below you can find diff between two objdumps.

[root@tpg-latitude5490 /]# cat bootx.linkers.diff 
--- bootx.BFD   2021-03-14 23:33:41.687446710 +0100
+++ bootx.LLD   2021-03-14 23:33:58.037519874 +0100
@@ -1,11 +1,12 @@
 
-systemd-bootx64.efi.BFD:     file format pei-x86-64
-systemd-bootx64.efi.BFD
-architecture: i386:x86-64, flags 0x00000133:
-HAS_RELOC, EXEC_P, HAS_SYMS, HAS_LOCALS, D_PAGED
-start address 0x0000000000003000
+systemd-bootx64.efi.LLD:     file format pei-x86-64
+systemd-bootx64.efi.LLD
+architecture: i386:x86-64, flags 0x00000132:
+EXEC_P, HAS_SYMS, HAS_LOCALS, D_PAGED
+start address 0x0000000000004000
 
-Characteristics 0x206
+Characteristics 0x207
+       relocations stripped
        executable
        line numbers stripped
        debugging information removed
@@ -14,11 +15,11 @@
 Magic                  020b    (PE32+)
 MajorLinkerVersion     2
 MinorLinkerVersion     36
-SizeOfCode             0000000000010000
-SizeOfInitializedData  0000000000006000
+SizeOfCode             000000000000f600
+SizeOfInitializedData  0000000000005e00
 SizeOfUninitializedData        0000000000000000
-AddressOfEntryPoint    0000000000003000
-BaseOfCode             0000000000003000
+AddressOfEntryPoint    0000000000004000
+BaseOfCode             0000000000004000
 ImageBase              0000000000000000
 SectionAlignment       00001000
 FileAlignment          00000200
@@ -29,9 +30,9 @@
 MajorSubsystemVersion  0
 MinorSubsystemVersion  0
 Win32Version           00000000
-SizeOfImage            0001c000
-SizeOfHeaders          00000400
-CheckSum               00022f3d
+SizeOfImage            0001b000
+SizeOfHeaders          00000370
+CheckSum               0001b531
 Subsystem              0000000a        (EFI application)
 DllCharacteristics     00000000
 SizeOfStackReserve     0000000000000000
@@ -47,7 +48,7 @@
 Entry 2 0000000000000000 00000000 Resource Directory [.rsrc]
 Entry 3 0000000000000000 00000000 Exception Directory [.pdata]
 Entry 4 0000000000000000 00000000 Security Directory
-Entry 5 0000000000013000 0000000a Base Relocation Directory [.reloc]
+Entry 5 0000000000000000 00000000 Base Relocation Directory [.reloc]
 Entry 6 0000000000000000 00000000 Debug Directory
 Entry 7 0000000000000000 00000000 Description Directory
 Entry 8 0000000000000000 00000000 Special Directory
@@ -59,367 +60,358 @@
 Entry e 0000000000000000 00000000 CLR Runtime Header
 Entry f 0000000000000000 00000000 Reserved

+Fangrui Song

Is this ld.lld (ELF) or lld-link (PE-COFF)? lld is a crunchgen style
linker which supports multiple binary formats.

I have seen two ways producing an EFI binary. Both are largely underspecified.

* ld.bfd -m $an_elf_emulation => objcopy -I elf64-x86-64 -O
efi-app-x86_64. This converts an ELF binary to a PE-COFF binary in a
largely underspecified way. It is not clearly how objcopy translates
the ELF sections and relocations.
* ld.bfd -m $an_pe_emulation (no objcopy). This should correspond to
lld-link. In lld-link, /fixed causes config->relocatable=false, which
eventually sets the IMAGE_FILE_RELOCS_STRIPPED flag.
  It seems that GNU ld does something a bit differently. bfd/peXXigen.c

/* For PIE, if there is .reloc, we won't add IMAGE_FILE_RELOCS_STRIPPED.
     But there is no .reloc, we make sure that IMAGE_FILE_RELOCS_STRIPPED
     won't be added. */
  if (! pe_data (ibfd)->has_reloc_section
      && ! (pe_data (ibfd)->real_flags & IMAGE_FILE_RELOCS_STRIPPED))
    pe_data (obfd)->dont_strip_reloc = 1;

One may need to debug ld.bfd to understand how it works.

Dnia poniedziałek, 15 marca 2021 23:21:11 CET Fāng-ruì Sòng pisze:

> +Fangrui Song
>
>> Hi,
>>
>> i'm trying to dig issue with building systemd-boot EFI images with
>> llvm/clang and linking with llvm/lld. In general building and linking
>> process works and i get my final objects. Unfortunately these EFI images
>> does not boot system. I noticed that .efi image is missing PE/COFF
>> relocations. When i changed linker to ld.bfd, voila everything is fine
>> and final efi image does boot system and have needed relocs. I tried

all[689/2941] ld.bfd -o src/boot/efi/systemd_boot.so -T /usr/lib64/gnuefi/
elf_x86_64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib64 /
usr/lib64/gnuefi/crt0-efi-x86_64.o src/boot/efi/disk.c.o src/boot/efi/
graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/
util.c.o src/boot/efi/boot.c.o src/boot/efi/console.c.o src/boot/efi/crc32.c.o
src/boot/efi/random-seed.c.o src/boot/efi/sha256.c.o src/boot/efi/shim.c.o -
lefi -lgnuefi /usr/lib64/gcc/x86_64-openmandriva-linux-gnu/10.2.1/libgcc.a
[690/2941] /usr/bin/clang -Isystemd-resolved.p -I. -I.. -Isrc/basic -I../src/
basic -Isrc/boot -I../

>> the ld.lld options related to relocations, but still no success. Thanks
>> for your help.
>>
>> Below you can find diff between two objdumps.
>>
>> [root@tpg-latitude5490 /]# cat bootx.linkers.diff
>> --- bootx.BFD 2021-03-14 23:33:41.687446710 +0100
>> +++ bootx.LLD 2021-03-14 23:33:58.037519874 +0100
>> @@ -1,11 +1,12 @@
>>
>> -systemd-bootx64.efi.BFD: file format pei-x86-64
>> -systemd-bootx64.efi.BFD
>> -architecture: i386:x86-64, flags 0x00000133:
>> -HAS_RELOC, EXEC_P, HAS_SYMS, HAS_LOCALS, D_PAGED
>> -start address 0x0000000000003000
>> +systemd-bootx64.efi.LLD: file format pei-x86-64
>> +systemd-bootx64.efi.LLD
>> +architecture: i386:x86-64, flags 0x00000132:
>> +EXEC_P, HAS_SYMS, HAS_LOCALS, D_PAGED
>> +start address 0x0000000000004000
>>
>> -Characteristics 0x206
>> +Characteristics 0x207
>> + relocations stripped
>>
>> executable
>> line numbers stripped
>> debugging information removed
>>
>> @@ -14,11 +15,11 @@
>>
>> Magic 020b (PE32+)
>> MajorLinkerVersion 2
>> MinorLinkerVersion 36
>>
>> -SizeOfCode 0000000000010000
>> -SizeOfInitializedData 0000000000006000
>> +SizeOfCode 000000000000f600
>> +SizeOfInitializedData 0000000000005e00
>>
>> SizeOfUninitializedData 0000000000000000
>>
>> -AddressOfEntryPoint 0000000000003000
>> -BaseOfCode 0000000000003000
>> +AddressOfEntryPoint 0000000000004000
>> +BaseOfCode 0000000000004000
>>
>> ImageBase 0000000000000000
>> SectionAlignment 00001000
>> FileAlignment 00000200
>>
>> @@ -29,9 +30,9 @@
>>
>> MajorSubsystemVersion 0
>> MinorSubsystemVersion 0
>> Win32Version 00000000
>>
>> -SizeOfImage 0001c000
>> -SizeOfHeaders 00000400
>> -CheckSum 00022f3d
>> +SizeOfImage 0001b000
>> +SizeOfHeaders 00000370
>> +CheckSum 0001b531
>>
>> Subsystem 0000000a (EFI application)
>> DllCharacteristics 00000000
>> SizeOfStackReserve 0000000000000000
>>
>> @@ -47,7 +48,7 @@
>>
>> Entry 2 0000000000000000 00000000 Resource Directory [.rsrc]
>> Entry 3 0000000000000000 00000000 Exception Directory [.pdata]
>> Entry 4 0000000000000000 00000000 Security Directory
>>
>> -Entry 5 0000000000013000 0000000a Base Relocation Directory [.reloc]
>> +Entry 5 0000000000000000 00000000 Base Relocation Directory [.reloc]
>>
>> Entry 6 0000000000000000 00000000 Debug Directory
>> Entry 7 0000000000000000 00000000 Description Directory
>> Entry 8 0000000000000000 00000000 Special Directory
>>
>> @@ -59,367 +60,358 @@
>>
>> Entry e 0000000000000000 00000000 CLR Runtime Header
>> Entry f 0000000000000000 00000000 Reserved
>>
>> -
>> -PE File Base Relocations (interpreted .reloc section contents)
>> -
>> -Virtual Address: 00004770 Chunk size 10 (0xa) Number of fixups 1
>> - reloc 0 offset 0 [4770] ABSOLUTE
>> -
>>
>> Sections:
>> Idx Name Size VMA LMA File off
>> >>
>> - 0 .text 0000ff20 0000000000003000 0000000000003000 00000400
>> 2**4 - CONTENTS, ALLOC, LOAD, READONLY, CODE
>> - 1 .reloc 0000000a 0000000000013000 0000000000013000 00010400
>> 2**0 + 0 .rela.dyn 00000fa8 0000000000000170 0000000000000170
>> 00000370 2**3>>
>> CONTENTS, ALLOC, LOAD, READONLY, DATA
>>
>> - 2 .data 00004460 0000000000014000 0000000000014000 00010600
>> 2**5 + 1 .text 0000f510 0000000000004000 0000000000004000
>> 00001400 2**4 + CONTENTS, ALLOC, LOAD, READONLY, CODE
>> + 2 .data 00004578 0000000000014000 0000000000014000 00010a00
>> 2**5>>
>> CONTENTS, ALLOC, LOAD, DATA
>>
>> - 3 .dynamic 000000f0 0000000000019000 0000000000019000 00014c00
>> 2**3 + 3 .dynamic 000000a0 0000000000019000 0000000000019000
>> 00015000 2**3>>
>> CONTENTS, ALLOC, LOAD, DATA
>>
>> - 4 .rela 00000fa8 000000000001a000 000000000001a000 00014e00
>> 2**3 - CONTENTS, ALLOC, LOAD, READONLY, DATA
>> - 5 .dynsym 000004e0 000000000001b000 000000000001b000 00015e00
>> 2**3 + 4 .dynsym 00000450 000000000001a000 000000000001a000
>> 00015200 2**3>>
>> CONTENTS, ALLOC, LOAD, READONLY, DATA
>>
>> 1. [systemd issue] - https://github.com/systemd/systemd/issues/19005
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev@lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Is this ld.lld (ELF) or lld-link (PE-COFF)? lld is a crunchgen style
linker which supports multiple binary formats.

This is ld.lld snapshot of 12.0.0 from 2021-03-05

I have seen two ways producing an EFI binary. Both are largely
underspecified.

* ld.bfd -m $an_elf_emulation => objcopy -I elf64-x86-64 -O
efi-app-x86_64. This converts an ELF binary to a PE-COFF binary in a
largely underspecified way. It is not clearly how objcopy translates
the ELF sections and relocations.
* ld.bfd -m $an_pe_emulation (no objcopy). This should correspond to
lld-link. In lld-link, /fixed causes config->relocatable=false, which
eventually sets the IMAGE_FILE_RELOCS_STRIPPED flag.
  It seems that GNU ld does something a bit differently. bfd/peXXigen.c

This is how a ld.bfd linking looks like :
[689/2941] ld.bfd -o src/boot/efi/systemd_boot.so -T /usr/lib64/gnuefi/
elf_x86_64_efi.lds -shared -Bsymbolic -nostdlib -znocombreloc -L /usr/lib64 /
usr/lib64/gnuefi/crt0-efi-x86_64.o src/boot/efi/disk.c.o src/boot/efi/
graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/
util.c.o src/boot/efi/boot.c.o src/boot/efi/console.c.o src/boot/efi/crc32.c.o
src/boot/efi/random-seed.c.o src/boot/efi/sha256.c.o src/boot/efi/shim.c.o -
lefi -lgnuefi /usr/lib64/gcc/x86_64-openmandriva-linux-gnu/10.2.1/libgcc.a
[690/2941] /usr/bin/clang -Isystemd-resolved.p -I. -I.. -Isrc/basic -I../src/
basic -Isrc/boot -I../

And here is the meson.build https://github.com/systemd/systemd/blob/main/src/
boot/efi/meson.build

/* For PIE, if there is .reloc, we won't add IMAGE_FILE_RELOCS_STRIPPED.
     But there is no .reloc, we make sure that IMAGE_FILE_RELOCS_STRIPPED
     won't be added. */
  if (! pe_data (ibfd)->has_reloc_section
      && ! (pe_data (ibfd)->real_flags & IMAGE_FILE_RELOCS_STRIPPED))
    pe_data (obfd)->dont_strip_reloc = 1;

One may need to debug ld.bfd to understand how it works.

This sounds like i should file a bug report for llvm/lld for missing
"feature".