[RFC] Running hermetic tests on baremetal Arm

As discussed in the LLVM Embedded Toolchains Working Group sync up, LLVM libc has limited testing with baremetal configurations.

Arm Toolchain fort Embedded (ATfE) enabled LLVM libc hermetic tests for baremetal Arm configurations using QEMU.

Reusing the experience and configuration/code from ATfE, here are some proof-of-concept patches to show what is required to be able to run the hermetic tests in the LLVM project itself:

Request for Comments

  1. What is the best place for the linker script?
  2. Is it OK to add the semihosting bits required for testing into the start.cpp itself or is there a better structure/place? Note that this is not a full semihosting implementation, though it may be a starting point for a future general semihosting library.
  3. Running the tests requires a number of additional options, adding all of them to the bot name may not be the best option - how to best structure these?
  4. The list of extra CMake options is rather long (based on ATfE configuration - maybe some of them can be still avoided) - is it OK to keep in the same builder file or better to split out?
  5. The workers would need qemu-system installed.

@petrhosek @michaelrj-google @Prabhuk please share your thoughts, thank you!

1 Like

Thanks for writing this up!

  1. I don’t think we have any examples of checked-in linker scripts yet. Putting it in with the startup code makes some sense, as would putting it in /config/baremetal/arm/. If we expect to have a lot of these it might even be best to create a new folder. I don’t have a strong opinion on this. I will say I’d like to avoid mixing licenses in the LLVM repository, sowe might need to make a custom linker script not based off of the picolibc one.
  2. Is semihosting only used for testing? If so it might be best to put it in the testing framework. Several of these functions seem like they don’t need to be run at startup, so I’d say they should probably go in /src/__support/OSUtil/baremetal.
  3. I’m not opposed to having moderately long bot names as long as they are descriptive, but if we start adding a lot of options it may get too long. Perhaps we want to have configuration flags that bundle several flags together? That way we can pick a couple relevant configurations and test those.
  4. I’m fine with it being long for now, we can look into cmake cache files later if it turns out to be a problem.
  5. I’m pretty sure installing qemu-system on a given buildbot would be doable, we already have the arm32 qemu buildbot.
1 Like