buildbot with -vectorize

I think that it would be a good idea to have a buildbot set up to run
the test suite with -vectorize. Could one of the existing machines do
this also?

Thanks again,
Hal

It took some time, but we have this now:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize

The first run shows that there are a couple of test-suite failures with the vectorizer:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL

Hal, interested to take a look?

Tobi

> I think that it would be a good idea to have a buildbot set up to
> run the test suite with -vectorize. Could one of the existing
> machines do this also?

It took some time, but we have this now:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize

The first run shows that there are a couple of test-suite failures
with the vectorizer:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL

Hal, interested to take a look?

Yes, thanks! (I already know there is a metadata issue I need to fix; I
should get to that today).

-Hal

> I think that it would be a good idea to have a buildbot set up to
> run the test suite with -vectorize. Could one of the existing
> machines do this also?

It took some time, but we have this now:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize

The first run shows that there are a couple of test-suite failures
with the vectorizer:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL

Hal, interested to take a look?

If I'm looking at this correctly, there are now no failures.
http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test

-Hal

I believe there are:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL

They are just automatically XFAILED after they occur a second time. :wink:

Tobi

>
>>> I think that it would be a good idea to have a buildbot set up to
>>> run the test suite with -vectorize. Could one of the existing
>>> machines do this also?
>>
>> It took some time, but we have this now:
>>
>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize
>>
>> The first run shows that there are a couple of test-suite failures
>> with the vectorizer:
>>
>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL
>>
>> Hal, interested to take a look?
>
> If I'm looking at this correctly, there are now no failures.
> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test

I believe there are:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL

They are just automatically XFAILED after they occur a second
time. :wink:

OIC... then that's not good...

So it seems that prior the build 108 the testing process always timed
out (it looks like it had a timeout of 20 minutes which was not long
enough). At build 108, it lists those applications as failing.

I am pretty sure that there were no x86 test-suite failures when I
committed the vectorizer. We'll need to investigate...

-Hal

I think that it would be a good idea to have a buildbot set up to
run the test suite with -vectorize. Could one of the existing
machines do this also?

It took some time, but we have this now:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize

The first run shows that there are a couple of test-suite failures
with the vectorizer:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL

Hal, interested to take a look?

If I'm looking at this correctly, there are now no failures.
http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test

I believe there are:

http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL

They are just automatically XFAILED after they occur a second
time. :wink:

OIC... then that's not good...

So it seems that prior the build 108 the testing process always timed
out (it looks like it had a timeout of 20 minutes which was not long
enough). At build 108, it lists those applications as failing.

Yes.

I am pretty sure that there were no x86 test-suite failures when I
committed the vectorizer. We'll need to investigate...

It may be due to the different platform (darwin). The list is sort so maybe you can try some of them on linux.

Tobi

>
>>>
>>>>> I think that it would be a good idea to have a buildbot set up
>>>>> to run the test suite with -vectorize. Could one of the existing
>>>>> machines do this also?
>>>>
>>>> It took some time, but we have this now:
>>>>
>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize
>>>>
>>>> The first run shows that there are a couple of test-suite
>>>> failures with the vectorizer:
>>>>
>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL
>>>>
>>>> Hal, interested to take a look?
>>>
>>> If I'm looking at this correctly, there are now no failures.
>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test
>>
>> I believe there are:
>>
>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL
>>
>> They are just automatically XFAILED after they occur a second
>> time. :wink:
>
> OIC... then that's not good...
>
> So it seems that prior the build 108 the testing process always
> timed out (it looks like it had a timeout of 20 minutes which was
> not long enough). At build 108, it lists those applications as
> failing.

Yes.

> I am pretty sure that there were no x86 test-suite failures when I
> committed the vectorizer. We'll need to investigate...

It may be due to the different platform (darwin). The list is sort so
maybe you can try some of them on linux.

Build 270 lists some of the nightly tests under improved and regressed
(because I enabled vectorizing comparisons). Is there a way to see the
relative timings? -- I thought that there was but I'm not seeing
anything to click to see a timing history.

-Hal

Yes, have a look here: http://llvm.org/perf/db_default/v4/nts/1437

Tobi

>
>>>
>>>>>
>>>>>>> I think that it would be a good idea to have a buildbot set up
>>>>>>> to run the test suite with -vectorize. Could one of the
>>>>>>> existing machines do this also?
>>>>>>
>>>>>> It took some time, but we have this now:
>>>>>>
>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize
>>>>>>
>>>>>> The first run shows that there are a couple of test-suite
>>>>>> failures with the vectorizer:
>>>>>>
>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL
>>>>>>
>>>>>> Hal, interested to take a look?
>>>>>
>>>>> If I'm looking at this correctly, there are now no failures.
>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test
>>>>
>>>> I believe there are:
>>>>
>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL
>>>>
>>>> They are just automatically XFAILED after they occur a second
>>>> time. :wink:
>>>
>>> OIC... then that's not good...
>>>
>>> So it seems that prior the build 108 the testing process always
>>> timed out (it looks like it had a timeout of 20 minutes which was
>>> not long enough). At build 108, it lists those applications as
>>> failing.
>>
>> Yes.
>>
>>> I am pretty sure that there were no x86 test-suite failures when I
>>> committed the vectorizer. We'll need to investigate...
>>
>> It may be due to the different platform (darwin). The list is sort
>> so maybe you can try some of them on linux.
>
> Build 270 lists some of the nightly tests under improved and
> regressed (because I enabled vectorizing comparisons). Is there a
> way to see the relative timings? -- I thought that there was but
> I'm not seeing anything to click to see a timing history.

Yes, have a look here: http://llvm.org/perf/db_default/v4/nts/1437

Thanks! How does it choose which set of runs to call the 'baseline'?

-Hal

That's the very first run. I normally ignore it as this probably was only run with one iteration and the timings are consequently not really reliable.

Tobi

>
>>>
>>>>>
>>>>>>>
>>>>>>>>> I think that it would be a good idea to have a buildbot set
>>>>>>>>> up to run the test suite with -vectorize. Could one of the
>>>>>>>>> existing machines do this also?
>>>>>>>>
>>>>>>>> It took some time, but we have this now:
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize
>>>>>>>>
>>>>>>>> The first run shows that there are a couple of test-suite
>>>>>>>> failures with the vectorizer:
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL
>>>>>>>>
>>>>>>>> Hal, interested to take a look?
>>>>>>>
>>>>>>> If I'm looking at this correctly, there are now no failures.
>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test
>>>>>>
>>>>>> I believe there are:
>>>>>>
>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL
>>>>>>
>>>>>> They are just automatically XFAILED after they occur a second
>>>>>> time. :wink:
>>>>>
>>>>> OIC... then that's not good...
>>>>>
>>>>> So it seems that prior the build 108 the testing process always
>>>>> timed out (it looks like it had a timeout of 20 minutes which
>>>>> was not long enough). At build 108, it lists those applications
>>>>> as failing.
>>>>
>>>> Yes.
>>>>
>>>>> I am pretty sure that there were no x86 test-suite failures
>>>>> when I committed the vectorizer. We'll need to investigate...
>>>>
>>>> It may be due to the different platform (darwin). The list is
>>>> sort so maybe you can try some of them on linux.
>>>
>>> Build 270 lists some of the nightly tests under improved and
>>> regressed (because I enabled vectorizing comparisons). Is there a
>>> way to see the relative timings? -- I thought that there was but
>>> I'm not seeing anything to click to see a timing history.
>>
>> Yes, have a look here: http://llvm.org/perf/db_default/v4/nts/1437
>
> Thanks! How does it choose which set of runs to call the 'baseline'?

That's the very first run. I normally ignore it as this probably was
only run with one iteration and the timings are consequently not
really reliable.

FWIW, it looks like almost all (if not all) of the vectorize
compile failures are due to other passes not handling pointer vectors
appropriately. For example, lib/Analysis/ValueTracking.cpp contains
this code:

427 if (SrcTy->isPointerTy())
428 SrcBitWidth = TD->getTypeSizeInBits(SrcTy);
429 else
430 SrcBitWidth = SrcTy->getScalarSizeInBits();

and this does not work for pointer vectors. In this case, the solution
(for the crash, which happens because getScalarSizeInBits returns 0 for
pointer vectors) seems obvious, although fixing it to not crash and
making sure it does the right thing are not equivalent.

I can submit bug reports for these things, but I'd appreciate someone
else helping with this. If you run with:
-mllvm -unroll-allow-partial -mllvm -unroll-runtime -mllvm -vectorize
-mllvm -bb-vectorize-req-chain-depth=4 (or 3 for real stress testing :wink:
) you'll pick up a lot more of these kinds of things.

If you run with -mllvm -bb-vectorize-no-gep -mllvm
-bb-vectorize-no-pointers you'll find that most of this stuff goes away.

-Hal

>
>>>
>>>>>
>>>>>>>
>>>>>>>>> I think that it would be a good idea to have a buildbot set
>>>>>>>>> up to run the test suite with -vectorize. Could one of the
>>>>>>>>> existing machines do this also?
>>>>>>>>
>>>>>>>> It took some time, but we have this now:
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize
>>>>>>>>
>>>>>>>> The first run shows that there are a couple of test-suite
>>>>>>>> failures with the vectorizer:
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL
>>>>>>>>
>>>>>>>> Hal, interested to take a look?
>>>>>>>
>>>>>>> If I'm looking at this correctly, there are now no failures.
>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test
>>>>>>
>>>>>> I believe there are:
>>>>>>
>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL
>>>>>>
>>>>>> They are just automatically XFAILED after they occur a second
>>>>>> time. :wink:
>>>>>
>>>>> OIC... then that's not good...
>>>>>
>>>>> So it seems that prior the build 108 the testing process always
>>>>> timed out (it looks like it had a timeout of 20 minutes which
>>>>> was not long enough). At build 108, it lists those applications
>>>>> as failing.
>>>>
>>>> Yes.
>>>>
>>>>> I am pretty sure that there were no x86 test-suite failures
>>>>> when I committed the vectorizer. We'll need to investigate...
>>>>
>>>> It may be due to the different platform (darwin). The list is
>>>> sort so maybe you can try some of them on linux.
>>>
>>> Build 270 lists some of the nightly tests under improved and
>>> regressed (because I enabled vectorizing comparisons). Is there a
>>> way to see the relative timings? -- I thought that there was but
>>> I'm not seeing anything to click to see a timing history.
>>
>> Yes, have a look here: http://llvm.org/perf/db_default/v4/nts/1437

Also, since you're running these on an x86_64 machine, and I think they
don't have unaligned vector load/stores, you should probably add -mllvm
-bb-vectorize-aligned-only to the target flags.

Thanks again,
Hal

Hi Hal,

FWIW, it looks like almost all (if not all) of the vectorize
compile failures are due to other passes not handling pointer vectors
appropriately.

unfortunately vectors of pointers were added without a systematic code audit to
ensure that all places handle such vectors correctly. Probably one good thing
to do is to have llvm-stress produce such vectors, and feed the output into all
of the LLVM optimizers.

Ciao, Duncan.

  For example, lib/Analysis/ValueTracking.cpp contains

[..]

Also, since you're running these on an x86_64 machine, and I think they
don't have unaligned vector load/stores, you should probably add -mllvm
-bb-vectorize-aligned-only to the target flags.

What about MOVUPS and MOVUPD?

Tobi

>
[..]

> Also, since you're running these on an x86_64 machine, and I think
> they don't have unaligned vector load/stores, you should probably
> add -mllvm -bb-vectorize-aligned-only to the target flags.

What about MOVUPS and MOVUPD?

Good point. Never mind. I suppose those can be used for the integer
vectors too.

Thanks again,
Hal

To my knowledge, integer vectors only make sense starting from AVX2. Before, integer vector operations are not really well supported on x86.

Tobi