[RFC] Emit relocation failure statistics

Subprojects LLD
1

Hi!

We are currently facing relocation overflow failures at Meta.
I’ve started the Massive Binary working group alongside @aeubanks to redesign a new code-model to address a lot of the challenges we have faced with the current available options.

( [RFC] Forming a Massive Binaries Working Group in LLD )

I wanted to better understand globally what the different relocation failures we are facing.
We have some simple parsing from the output (slightly patched to be a bit more ergonomic) to categorize the relocation failure types however we limit lld error messages to 100

ld.lld: error: too many errors emitted, stopping now (use --error-limit=0 to see all errors)
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

We’ve done this to help reduce the amount of bytes we send to various logging platforms however now I lose visibility into the complete view of all the various relocations I would have faced since often the first 100 are occupied by a single type (i.e. gcc_except_table → data)

I would like to propose an option for LLD to emit a structured file with enough information for us to categorize all the failures (could be scoped to relocation failures to start).

Here is a proposed schema merely to demonstrate the type of data I am looking to surface.

{
  "version": "1.0",
  "build_context": {
    "target_triple": "x86_64-unknown-linux-gnu",
    "linker_version": "LLD 19.0.0",
    "command_line_args": ["-flavor", "gnu", "-o", "bin/huge_app", "..."],
    "total_errors_encountered": 14208
  },
  "relocation_summary": {
    "total_overflows": 14195,
    "unique_overflow_types": 3
  },
  "failures": [
    {
      "reloc_type_id": "R_X86_64_PC32",
      "reloc_type_name": "R_X86_64_PC32",
      "count": 12450,
      "description": "32-bit PC-relative relocation overflow",
      "section_pair": {
        "source_section": ".gcc_except_table",
        "target_section": ".text"
      },
      "overflow_characteristics": {
        "max_negative_displacement": -34359738368,
        "max_positive_displacement": 4294967296,
        "reloc_limit_bits": 32,
        "is_signed": true
      },
      "samples": [
        {
          "source_file": "path/to/folly/experimental/huge_file.cpp.o",
          "source_symbol": "void folly::detail::complex_template_instantiation<int>()",
          "source_offset": "0x00001a4c",
          "target_symbol": "typeinfo for std::exception",
          "target_section": ".rodata",
          "calculated_displacement": -34359738368
        }
      ]
    },
    {
      "reloc_type_id": "R_AARCH64_CALL26",
      "reloc_type_name": "R_AARCH64_CALL26",
      "count": 1745,
      "description": "26-bit branch instruction offset out of range (+/-128MB)",
      "section_pair": {
        "source_section": ".text",
        "target_section": ".text"
      },
      "overflow_characteristics": {
        "max_positive_displacement": 157286400,
        "reloc_limit_bits": 26,
        "is_signed": true
      },
      "samples": [
        {
          "source_file": "path/to/engine/core.cpp.o",
          "source_symbol": "engine::run_main_loop()",
          "source_offset": "0x0004f210",
          "target_symbol": "fbcode::logging::log_verbose_impl()",
          "target_section": ".text",
          "calculated_displacement": 157286400
        }
      ]
    }
  ]
}