LLVM Security Response Group public sync-ups meeting minutes

LLVM Project
32

Meeting minutes for the 19th of November 2024 meeting

  • There was a question about the discussions that took place at the LLVM dev meeting
    security group round table a few weeks ago.
    The meeting minutes of that round table have been published at LLVM security group roundtable notes.

    We talked about probably the most notable discussion being that some projects using clang/llvm will probably start to depend more on some of the security hardening features. When they do so, gaps or bugs in the implementation of such security hardening features will be far more critical security issues for these projects.

    We discussed how the implementation of security hardening features could be tested well. Today, the automated testing that exists is probably mostly limited to:

    • unit test/regression tests: they test that the security hardening feature does generate code in line with what is expected from the feature, but only for a small number of small cases. Presumably, these tests would catch it if a security hardening feature would completely stop working everywhere. But they are unlikely to catch if a hardening feature stops working in only some circumstances.
      • One of the attendees with years of experience working on security hardening features shares that it happens regularly that there are regressions in hardening features that are not found by automatic tests.
    • It’s unclear if there are bots who test some of the security hardening features by enabling them when building and running larger benchmarks and test-suites. Even if they do exist, this only tests if the security hardening feature doesn’t completely break execution. They do not test whether the security hardening feature actually protects the binary as intended.

    Manual pen testing could be helpful to find gaps in security hardening features. But presumably, the manual aspect results in not quickly finding regressions.
    @kbeyls mentioned that his EuroLLVM talk from earlier this year specifically covers this topic, proposing to build a binary analysis tool that can verify if generated code has the properties that are expected from the security hardening feature.
    A prototype is available at GitHub - kbeyls/llvm-project at bolt-gadget-scanner-prototype. More details can also be found in this RFC.
    A first patch to upstream this prototype tool was recently posted as PR115330.

  • An observation was shared that the LLVM security group is expected by some to guarantee good security in all of LLVM. However, the LLVM security group’s focus is much more narrow: to make sure it is possible to responsibly disclose security issues in llvm. The security group does not actively explore how to make LLVM more secure.

    We brainstormed on what we could do to address the issue that some people assume the LLVM security group will do more than what it actually does. The concrete ideas we came up with are:

    • Rename the group to “LLVM security response group”, and update LLVM’s documentation accordingly. @smithp35 volunteers to create a PR for this. @kbeyls volunteers to help with getting this PR progressed appropriately.
    • The documentation for the LLVM security group at LLVM Security Response Group — LLVM 22.0.0git documentation states 6 goals for the group at the top of the document. In practice, the group actively works on the first 5 listed goals, but not on the 6th. @smithp35 volunteers to create a PR to remove item 6 there.

  • The LLVM security response group needs to start preparing for a transparency report as we’re near the end of the year. Since we moved to github for security issue reporting, we’ll need to do more manual work to create the transparency report. The group has a shared document to prepare the transparency report. @kbeyls volunteers to write most of the report. But we still need a volunteer to write entries for the issues related to LLVM project infrastructure/supply chain security.

  • The members of the LLVM security response group present in the call discussed open issues, how to best progress them and took actions on progressing them. Thanks to @smithp35, @ahmedbougacha , @Tulio_Magno_Quites_M , @kbeyls for taking actions!

@artur @mrragava @DimitryAndric @emaste @gburgessiv @cuviper @ormris @nikhilg @waffles_the_dog @serge-sans-paille @whuhn @yroux