LNT usage

Project Infrastructure LLVM Dev List Archives
1

While porting my backends to llvm-3.2, I found a few places where the
optimizers could have performed better. I believe the mainstream targets can
also benefits from my tweaks. But before upstreaming my changes, I would like
to quantify their merits on other applications --- not just my domain specific
codes. In a word, it seemed the right time for me to start using LNT :slight_smile:

I followed the LNT quickstart guide, and somehow got some results, but have
trouble using them. How are the other developers using lnt ? What is the
workflow for comparing a patched lllvm to the baseline version ?

I am mainly interested in the tests' execution time.

1. I am bit confused by the 'lnt runtest' output:

$LNT_TOP/sandbox/bin/lnt runtest nt \

  --sandbox=$LNT_TOP/SANDBOX3 \
  --cc=$LLVM_BIN/clang \
  --cxx=$LLVM_BIN/clang++ \
  --test-suite=$LLVM_SRCS/projects/test-suite \
  --llvm-src=$LLVM_SRCS \
  --llvm-obj=$LLVM_BUILD

2013-03-11 18:41:22: checking source versions
2013-03-11 18:41:25: scanning for LNT-based test modules
2013-03-11 18:41:25: found 0 LNT-based test modules
2013-03-11 18:41:25: using nickname: 'm0__clang_DEV__x86_64'
2013-03-11 18:41:25: starting test in
'.../LLVM/LNT/SANDBOX3/test-2013-03-11_18-41-22'
2013-03-11 18:41:25: configuring...
2013-03-11 18:41:34: executing "nightly tests" with -j1...
2013-03-11 19:00:42: executing test modules
2013-03-11 19:00:42: loading nightly test data...
2013-03-11 19:00:42: capturing machine information
2013-03-11 19:00:42: generating report:
'.../LLVM/LNT/SANDBOX3/test-2013-03-11_18-41-22/report.json'
2013-03-11 18:41:22: submitting result to dummy instance
No handlers could be found for logger "lnt.server.db.migrate"
Importing 'tmpItDGOn.json'
Import succeeded.

Processing Times

2
  1. Running lnt several times on the same unmodified clang+llvm binaries gives
    different results (execution time can vary wildly : ~200%). I am running lnt
    on linux/x86_64. I tried to deactivate the cpufreq thing, the machine was not
    loaded, but this did not change. Is there any way to run the tests multiple
    times and use statistics to get reproducible numbers (within a confidence
    interval) ? Or to handle the stats at the ‘lnt import’ stage ? Or in the web
    display ? In the web display, how to use the ‘run’ and ‘order’ fields ?

Run “lnt runtest nt --help”. = ).

Michael

3

:slight_smile:

Done that several times — and even went in the lnt source code — but I missed the ‘–multisample’ each time. That’s probably what I need.

Thanks Michael,