Add gfortran tests to llvm-test-suite

[lenary]

I was responsible for doing this for the GCC C torture suite, and I’m glad to see us taking a similar approach for the gfortran test suite! I think it’s good to share test suites as much as possible, where they are testing issues that any toolchain may run into.

@tarunprabhu

Of these, a number of tests have been disabled because they cannot be built. Some fail to build because they use features not yet implemented in flang and error message says as much. Others may also be a result of unimplemented features but that is not apparent from the error message. Some may be failing because they use unsupported (non-standard) extensions.

Looking at what I did for C, we have two separate lists:

• UnsupportedTests which either rely on GCC-specific extensions that we have not implemented, or rely on undefined behaviour, or our CMake configuration doesn’t support them correctly (due to not extracting arguments correctly)
• FailingTests which are tests that should be supported, but are not by clang (which I almost certainly should have filed issues about, but have not).

I realise that clang, when I added these, had a different proportion of UnsupportedTests vs FailingTests than flang will have, but making this distinction is useful (even moreso if we had a good xfail system - maybe I’ll look at that again for these), and i think should make it clear which list you want to make smaller, and which one you have no intention of making smaller.

@tarunprabhu

It may be useful to also enable the “compile” tests.

For C, we did not, because the test suite is really about compilation+execution. We did say we’d revisit this and never got around to it.

I think broadly the llvm_test_executable_no_test(target, file) call in gcc_torture_execute_test means “check compilation doesn’t fail”, and llvm_test_run(…) means “also try running this test (maybe with arguments)”, when generating the underlying lit script used during testing. Note we never called llvm_test_verify because we never had reference outputs for the GCC torture suite. cmake/modules/TestFile.cmake explains a bit more about what is going on with this, but not quite enough. There’s no function for adding an XFAIL: ... line to the lit script but it would not be hard to add, and it would also be useful for some of the FailingTests in the C torture suites.

@tarunprabhu

Longer term, it may be useful to actually check for the appropriate error messages. This would involve replacing the DejaGNU annotations in the tests with LLVM-specific ones.

We do not want to make modifications to the GNU-licensed files in llvm-test-suite, instead, we want to just exclude stuff that doesn’t work, or ignore annotations we don’t understand. I think it’s fine we don’t check error messages, to be honest, but I can see the advantage in “check we reject this program like the relevant gnu toolchain does”. Maybe this is something for later, and it’s best to start with just the execution tests which have a clearer path to pass/fail.

One of my hopes is that we could get the llvm-test-suite so it’s usable to test gcc/gfortran, and i wonder if it would be possible to do something similar for the dejagnu system in the gcc/gfortran repositories so those tests can be run with clang/flang eventually. This will require some cross-project coordination, but I think this kind of cross-testing is good, actually (which is why I added the C torture suite).

I’m going to do a review of the final patch in the series. When i did the C torture suite patches, I think the patch landing all the sources of the tests wasn’t reviewed, only the CMake was, which seems like a reasonable approach here too.