test-suite RunSafely.sh remote execution with test copy

Project Infrastructure LLVM Dev List Archives
1

Hello everyone,

I am working to run the LLVM Nightly Testsuite on embedded arm devices. Not all of these devices can mount NFS to share a common directory tree. To mitigate this I have patched test-suite/RunSafely.sh (attached) to create a temporary directory, run the test remotely, collect the output, and cleanup after itself. The patch breaks compatibility with the current remote device model in the following ways:

  1. uses scp/rcp to copy files from the host running test-suite instead of assuming an identical tree on the remote device.

  2. expects the remote device to have the mktemp command to create the temporary workspace (usually under /tmp/*)

  3. expects the remote device to have timeit installed in its path. If timeit were compiled statically I could scp the one passed into RunSafely.sh to the destination. Hosts with a different libc than the tested compiler (Android, embedded uClibc devices) cannot run the dynamically-linked timeit.

  4. It’s a bit slower due to all the network copies for the entire test suite.

I’d appreciate feedback and suggestions as to how I can rework this model for inclusion upstream. I’m sure there are others out there that would find this kind of testing useful. I’m fine with a new remote-copy mode that can be passed in via lnt nt command line invocation.

I hope others find this useful.

-Brian

RunSafely-rhost-copy.patch (2.55 KB)

2

Have you tried sshfs.
I does much the same as nfs mount but uses ssh instead.

3

Hi James,

Thanks for the feedback. Yes I did look into sshfs and It’s a great choice for devices running Linux distributions. Unfortunately some of our devices are Android based and have no plans for Linux support. I started examining cross-compiling libfuse for Android but ran into complications there due to the very limited nature of the device’s root filesystem libraries, binaries, and compiled kernel options.

It was much simpler cross-compiling dropbear + scp for Android which lead to the relatively simple scp patch in RunSafely.sh.

I will take another stab at sshfs to see if there may be a way to get it working on Android with minimal pain.

I’d be curious to know if others doing similar work rely mostly on NFS, SMB, sshfs or some other network filesystem to accomplish remote execution testing.

-Brian

4

So I think this is a great idea, and something we really need. NFS is bad for this sort of testing because a NFS glitch can cause a test to hang. I think any transparent pretend filesystem solution is complicating things by interleaving the benchmark with the network file access. Explicit rsync or scp are the way to go here. A few questions:

  1. How much slower is it?
  2. How do you deal with tests where there are a bunch of tests in the same location? Shootout for example.
  3. Why did you choose to use a device native timeit?
  4. Do all the test currently pass?