Adding an execution result keyword in LIT

Project Infrastructure LLVM Dev List Archives
1

Hey all,

I am working on adding an execution result keyword such as PASSED, FAILED, XFAILED, etc. to LIT to indicate the result for each RUN execution. For example,

: ‘RUN: at line 1’ (PASSED); /home/anshil/amd-workspace/llvm-project/build/bin/llc -march=amdgcn -mcpu=tahiti -verify-machineinstrs < /home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll | /home/anshil/amd-workspace/llvm-project/build/bin/FileCheck /home/anshil/amd-workspace/llvm-project/llvm/test/CodeGen/AMDGPU/inline-calls.ll

I was wondering what would be the most appropriate place to implement this? I was thinking somewhere under parseIntegratedTestScriptsCommands(source_path, keywords) in TestRunner.py but I am open to suggestions.

Best,

Anshil Gandhi

2

Thanks! I've been wanting this annotation for a while now :slight_smile:

I can help review something, but I wouldn't be able off-hand to make a
suggestion on how to do it!

3

The "RUN: at line 1" part is printed by prepending an null command (a
colon) to the command line, which personally found to be a rather
strange implementation. Since PASSED/FAILED depends on the result, it
could at most be done by appending a command. Maybe both should be
printed in python itself, somewhere in TestRunner.py executeShCmd.
This executes all RUN-lines at once, with a single time-out. Something
that executes after each RUN line would need to be added. However,
printing the RUN-line before executing the command has the theoretical
advantage that one can could see where the current execution is stuck,
which may not matter that much since typically output is printed only
after it is known that the command failed.

Michael

4

Hey Michael,

Thanks for your response! executeShCmd() is never being called when I run ./bin/llvm-lit -v <test-file>.

I am thinking of somehow interacting with the subprocess.Popen(…) in lit/util.py::executeCommand(…) since this is where the script is executed. I need to obtain the execution result status somehow, this is the tricky part.

Best,

Anshil

5

I wonder if I could rely on the exit status of each child process and conclude PASSED or FAILED based on that. Please let me know if this is the case.

Anshil