Below is a high-level summary of the performance of flang on the gfortran test suite:
| Category | Count | Notes |
|---|---|---|
| Total | 7878 | |
| Viable | 7688 | Tests that are not excluded on the platform (Linux, x86_64) or explicitly unsupported because of unsupported extensions etc. |
| Passing | 5584 | |
| Failing | 2104 |
A rough count of the causes of the failures are provided below. These are deliberately broad categories just so we can get a rough idea of how things stand. Some of the categories disproportionately affect certain categories of tests. For instance, most of the scanning and parsing errors affect the OpenMP and OpenACC tests.
| Cause | Count | Notes |
|---|---|---|
| Not yet implemented assertion | 858 | |
| Segmentation fault | 100 | These all seemed to be at compile-time. The runtime never segfaulted |
| Could not scan | 22 | |
| Could not parse | 161 | |
| Unknown commandline argument | 91 | Some of these are limitations with the test suite which will be addressed. |
| Undeclared identifier | 31 | These seemed to all be related to C interoperability and something missing in the header. |
| Undefined reference | 34 | These seemed to be related to math functions missing from the runtime for the most part |
| Semantic Errors | 239 | |
| Linker Error | 7 | This is due to the -flto not being handled correctly in the test suite. |
| Runtime error | 14 | The runtime raised an error |
| Terminated by signal | 4 | These are probably timeouts but may be a result of an issue in the runtime |
| Others | 543 | These are usually mismatches between the expected and actual test results |
I can provide a more detailed breakdown if required at the flang community call.
[EDIT: Fix table header]