around the benchmark.
I accept Matthias` proposal, and I propose that the above be used for wherever "really fixing" the FP problems with a test is beyond a reasonable effort+time level. This email is to check acceptance [or lack thereof] of the above.
Regards,
Abe
From: "Abe Skolnik" <a.skolnik@samsung.com>
To: "Matthias Braun" <mbraun@apple.com>
Cc: "Hal Finkel" <hfinkel@anl.gov>, "Renato Golin" <renato.golin@linaro.org>, "llvm-dev" <llvm-dev@lists.llvm.org>
Sent: Wednesday, October 12, 2016 11:00:39 AM
Subject: Matthias` suggestion for "test-suite" tests that are broken at "-Ofast" and are difficult to "repair"> I don't find it surprising that some applications do not work
> properly with -ffast-math and I think we
> have to accept that fact. I think it is valid to skip those tests
> in the test-suite when a fast math
> flag combination is used (after making sure there is no easy way to
> make the test more robust).> I would add a "TEST_SUITE_FAST_MATH" variable in the cmake build
> (possibly scanning CFLAGS for -ffast-math/-Ofast
> to determine the default value) and then adding if(NOT
> TEST_SUITE_FAST_MATH) ... endif()
around the benchmark.I accept Matthias` proposal, and I propose that the above be used for
wherever "really fixing"
the FP problems with a test is beyond a reasonable effort+time level.
This email is to check
acceptance [or lack thereof] of the above.
This makes sense to me. I think we should set this up so that the tests are enabled by default, and then we specifically disable those tests for which we've determined that is the best practical solution.
-Hal
With -ffast-math, all bets are off, and it makes no sense to check
against good values, I agree.
But the benchmarking mode could be run with -ffast-math, and you'd
still want to make sure it works to some level. I mean, we run SPEC
with -ffast-math and it's not that bad, so it should be doable.
But from a test point of view, what sense does it make to run with
-ffast-math if you'll discard all "problematic" FP tests? Why run the
tests at all in that config if they're not meaningful?
I'm not against marking them off for fast-math, I'm just honestly
trying to understand what the point of it is... simply put: "what is
the meaning of saying all integer and some non-previously-failing FP
tests `pass` with -ffast-math"?
cheers,
--renato
From: "Renato Golin" <renato.golin@linaro.org>
To: "Hal Finkel" <hfinkel@anl.gov>
Cc: "Abe Skolnik" <a.skolnik@samsung.com>, "llvm-dev" <llvm-dev@lists.llvm.org>, "Matthias Braun" <mbraun@apple.com>
Sent: Wednesday, October 12, 2016 12:22:12 PM
Subject: Re: Matthias` suggestion for "test-suite" tests that are broken at "-Ofast" and are difficult to "repair">> I accept Matthias` proposal, and I propose that the above be used
>> for
>> wherever "really fixing"
>> the FP problems with a test is beyond a reasonable effort+time
>> level.
>> This email is to check
>> acceptance [or lack thereof] of the above.
>
> This makes sense to me. I think we should set this up so that the
> tests are enabled by default, and then we specifically disable
> those tests for which we've determined that is the best practical
> solution.With -ffast-math, all bets are off, and it makes no sense to check
against good values, I agree.But the benchmarking mode could be run with -ffast-math, and you'd
still want to make sure it works to some level. I mean, we run SPEC
with -ffast-math and it's not that bad, so it should be doable.But from a test point of view, what sense does it make to run with
-ffast-math if you'll discard all "problematic" FP tests? Why run the
tests at all in that config if they're not meaningful?I'm not against marking them off for fast-math, I'm just honestly
trying to understand what the point of it is... simply put: "what is
the meaning of saying all integer and some non-previously-failing FP
tests `pass` with -ffast-math"?
I think it depends on what "problematic" means. I want to keep those that we can check by just using a larger (but still reasonable) tolerance, smaller number of iterations, etc. We should only disable the ones for which there is not another reasonable solution.
-Hal
Right, that's ok. Means we'll investigate each individual test and
make sure the tolerances, sampling or disabling will be done in that
order, if all previous attempts failed.
cheers,
--renato