Thank you for the explanation George. I have too came to believe that a
general all-encompassing solution would be incredibly complicated. Our
(lldb) use case is in many ways similar to compiler-rt (we want to test
code which was not compiled for the host), but then it also has it's own
"thing" (we have *two* processes/binaries, one debugging another,
theoretically they don't even have to have the same bitness).
I am glad you were able to make your use case work. I'll probably try
creating something a bit custom for lldb, when I actually get around to it.
regards,
pavel
Hi Pavel,
I was thinking about a generic proposal for a while,
but then I’ve realized that a much simpler heuristic suffices for all of
my needs:
- Prepend all commands which should execute on the device with %run,
which runs a wrapper python script
- Inside that script use a simple regexp to find all paths
- sync all paths to the device
- run the executable on the device
- sync all paths form the device back to host
>
> Hello George,
>
> I would be very much interested in a (generic) solution that would allow
> running tests on a remote host/device. My intended use case for this is
> lldb+android, and I am mostly interested in googletest-style tests.
I’m not sure about those, all my problems were in LIT tests.
> In
> fact, I proposed something vaguely similar to this (but limited to
gtest)
> last year <http://lists.llvm.org/pipermail/llvm-dev/2017-May/113370.html
.
> However, due to some internal reshuffles, this idea never got past the
> prototype stage. Nonetheless, I would still like do something like this
> eventually (even better if someone else does it for me 
).
>
> I would love to hear more about your proposal, as my head is currently
> filled with questions about how this would work. (I admit I don't know
> anything about how libFuzzer is tested.) It's not clear to me whether
you
> intend to implement a general purpose remote test execution framework,
or a
> solution tailored to the needs of libFuzzer.
Seemed generic enough for most of compiler-rt.
> Do you automatically handle
> copying the inputs to the remote target? How do you identify what the
> inputs are?
Yes, the simple heuristic above seems to work here.
> Do you copy over any llvm tools executed in the run lines?
I copy the command directly after %run wrapper, but not FileCheck/etc.
%run syncs it over to the device.
> What
> about their dependent libraries? etc.
Dynamically linked? Mostly we don’t have any, and for those tests where
we do,
some rpath substitution magic is required.
> Maybe if you could give an example of a typical test you intend to
support
> and how it would work, a lot of these questions would be answered. If
this
> turns out to be something that could be reused for our use case, I
would be
> willing to chip in to help make it happen.
Basically, any of libFuzzer tests with %run prepended, e.g.
https://reviews.llvm.org/D45235
George
>
> regards,
> pavel
>
>
>> Hi,
>
>> Tests in compiler-rt run under both iOS simulators and Android devices,
> which requires “wrapping” the run line with the “adb” command for
Android,
> and adding a prefix / setting some environment variables for simulators/
>> In case of Android this is done by changing the compilation command to
> produce a wrapper which already calls “adb”, and in case of iOS
simulators
> by explicitly adding the “%run” command in front of run lines.
>> The solution is far from perfect though: many tests (e.g. libFuzzer)
> extensively rely on using arbitrary shell commands to create and
manipulate
> temporary text files.
>> Performing those commands in a different environment is done in a very
> ad-hoc manner: from what I see, for sanitizer tests those temporary
files
> are then usually references in the environment variables present in the
run
> command, which are then parsed to carry those files into the executable
> environment.
>> Doing this for libFuzzer would be very difficult and would require a
lot
> of modifications to test.
>
>> I would prefer a more transparent approach: if LIT configuration was to
> provide a hook to perform arbitrary replacements/wrapping on each of the
> executed lines, the wrapping could be performed without modifying tests
at
> all.
>
>> My implementation preference would be adding a method `def wrap(self,
> s)` (str -> str) to the format class (lit/formats/*), and then using it
(if