ELF header does not hold big modules

Hello,

I have a core dump that LLDB cannot open because it contains more than 64K sections. The “readelf” utility gives me the output in the end of this message. It seems that the actual number of program headers and the index of string table section are written into very first section since they do not fit in 16 bits.

The “natural” way to deal with this problem would be to change the types of fields in ELFHeader struct from 16 to 32 bits (or should I go all the way and do it 64? in case the core dump is bigger than 4GB…) and deal with the problem in a single place where we parse the header - the ELFHeader::Parse().

Objections? Suggestions? Advices?

Hmm… I am not sure that the DataExtractor we pass down there would let me read that much past in the file - I have impression that we give only first 512 bytes there, but I might be mistaken…

Thanks,

Eugene

ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: CORE (Core file)
Machine: Advanced Micro Devices X86-64
Version: 0x1
Entry point address: 0x0
Start of program headers: 64 (bytes into file)
Start of section headers: 16880770624 (bytes into file)
Flags: 0x0
Size of this header: 64 (bytes)
Size of program headers: 56 (bytes)
Number of program headers: 65535 (106714)
Size of section headers: 64 (bytes)
Number of section headers: 0 (106716)
Section header string table index: 65535 (106715)

Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
000000000001a0dc 0000000000000000 106715 106714 0
[ 1] note0 NOTE 0000000000000000 005b2ff0
00000000000b2e38 0000000000000000 A 0 0 1
[ 2] load PROGBITS 000000006a400000 00665e28
0000000000001000 0000000000000000 A 0 0 1
[ 3] load PROGBITS 000000006a401000 00666e28

Hello Eugene,

Hello,

I have a core dump that LLDB cannot open because it contains more than 64K
sections. The "readelf" utility gives me the output in the end of this
message. It seems that the actual number of program headers and the index of
string table section are written into very first section since they do not
fit in 16 bits.

The "natural" way to deal with this problem would be to change the types of
fields in ELFHeader struct from 16 to 32 bits (or should I go all the way
and do it 64? in case the core dump is bigger than 4GB...) and deal with
the problem in a single place where we parse the header - the
ELFHeader::Parse().

Objections? Suggestions? Advices?

Sounds like a plan. By "all" I presume you mean "all fields that refer
to section counts or indexes". I don't see a reason the change the
size of the e.g. e_type field. I think going 32 bit will be enough,
particularly as the "fallback" field you are reading this from is only
32 bit wide anyway, and so you would still have to touch this if we
ever cross that boundary. (And we are really talking about 4 billion
sections, not a 4GB core file, right?)

Hmm... I am not sure that the DataExtractor we pass down there would let me
read that much past in the file - I have impression that we give only first
512 bytes there, but I might be mistaken...

The reason we initially read only the first few bytes of the file, is
to be able to make a quick decision as to whether this is likely to be
a valid elf file (see call to ObjectFileELF::MagicBytesMatch in
GetModuleSpecifications). After that, we extend the buffer to the
whole file. It looks like this currently happens before parsing the
ELF header, but I don't see a reason why it would have to stay that
way.

cheers,
pavel

By “all” I presume you mean “all fields that refer to section counts or indexes”.

Correct. I mean exactly 3 fields: e_phnum, e_shnum, and e_shstrndx that could be redirected to section #0.

Found this on the web:

e_shnum
    This member holds the number of entries in the section header
    table. Thus the product of e_shentsize and e_shnum gives the section
    header table's size in bytes. If a file has no section header table,
    e_shnum holds the value zero.

    If the number of sections is greater than or equal to SHN_LORESERVE
    (0xff00), this member has the value zero and the actual number of
    section header table entries is contained in the sh_size field of
    the section header at index 0. (Otherwise, the sh_size member of the
    initial entry contains 0.)

See the second paragraph. I am guessing the same thing would need to happen for program headers? I would check the various ELF file parsers in LLVM and see if they handle this correctly?

Greg

Yes, that’s what happens to all 3 fields. I already fixed LLDB, Pavel reviewed the change.