I'm testing a bare-metal ARM toolchain, and I've hacked up my local copy of lit to get it to run libcxx tests on qemu. I wanted to pick your collective brains to see if there was a better way of doing this.
What I have implemented is here (don't take this as a formal review request, we can do that later once we figure out the best direction to go):
lit part: ⚙ D5014 Add QEMU args to lit
libcxx part: ⚙ D5015 Add QEMU testing to libcxx's lit config
Talking with Dan Albert earlier, he pointed out an existing review (⚙ D4594 [libcxx] Add support for Android targets to lit.cfg.) where he has implemented something vaguely similar for Android. Looking at that, I've realized that maybe it makes sense to make an interface for the different actions of:
* Compiling the test
o Maybe the compiler isn't on the machine that runs the test?
o Maybe the compile jobs could be distributed to different machines?
* Running the test
o Perhaps the test needs to be sent out to a remote device?
o Perhaps the test can be scheduled on more than one remote device?
Then that interface could be implemented in a couple of generic ways:
* scp & ssh for remote hosts (or QEMU with an OS image)
* Wrapper around Android's adb
* Thin wrapper around "run it in this machine's shell" (what we currently have)
What are your thoughts on this? Is this worth pursuing? What other use-cases can we support by structuring LIT this way? I don't yet have a concrete proposal for what this would look like, but I can throw one together if there's interest.