[Classic Flang] RFC: Adding release_12x branch to flang-compiler/classic-flang-llvm-project

Hi,

I have created a candidate release_12x branch in classic-flang-llvm-project. It is a rebase of all commits from release_11x since 43ff75f2, minus the DebugInfo additions that already exist in upstream LLVM 12. To build Classic Flang with this branch, we also need some minor changes to Classic Flang itself (see PR #1010). I have tested the build on both X86 and AArch64, and made sure all regression tests pass.

Any help from the community to review and test this branch would be greatly appreciated. Once the final LLVM 12.0.0 is tagged, I will rebase the branch and pull it into classic-flang-llvm-project, if there is no objection from the community.

Thanks,

Hi Bryan,

Thanks a lot for your efforts with creating this branch. I checked out both your classic-flang-llvm-project/release_12x branch and the code in PR #1010, that you mentioned. They both built fine for me with gcc-10, but I got some errors from the tests I ran. I expect we should also see them in CI, once the release_12x branch is pulled into the origin repo.

Classic-flang-llvm-project errors:

Failed Tests (4):
Clang :: Driver/flang/classic-flang-must-preprocess.F
Clang :: Driver/flang/classic-flang-must-preprocess.F95
Clang :: Driver/flang/classic-flang.f
Clang :: Driver/flang/classic-flang.f95

make install went fine despite the test failures.
With the new llvm-12 I ran the flang project tests and got this error:

Failed Tests (1):
Flang :: f90_correct/lit/array_constructor_nested_implied_do.sh

This comes from a test I just added recently in PR#1004, so perhaps it just needs a slight adjustment to the newer llvm tools. I can check this.

I am more worried about the llvm project failures. We do run clang and llvm tests on every PR, but when pulling the branch as a whole we won’t see them until the first PR to the new branch is submitted for review (just like was the case with release_11x).
Are you able to reproduce these failures on your side?

Best regards,

Hi,

I have created a candidate release_12x branch in classic-flang-llvm-project. It is a rebase of all commits from release_11x since 43ff75f2, minus the DebugInfo additions that already exist in upstream LLVM 12. To build Classic Flang with this branch, we also need some minor changes to Classic Flang itself (see PR #1010). I have tested the build on both X86 and AArch64, and made sure all regression tests pass.

Any help from the community to review and test this branch would be greatly appreciated. Once the final LLVM 12.0.0 is tagged, I will rebase the branch and pull it into classic-flang-llvm-project, if there is no objection from the community.

Thanks,


Bryan Chan


flang-dev mailing list
flang-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev

Mobica is a global software services company, delivering and enabling technologies that transform business outcomes for the leading brands in Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow UK with offices across Europe and the US, our established technical and delivery excellence in high quality software engineering drives success for our multinational customers on every continent, every day.
Find out more at Mobica.com

Mobica Limited is a limited company registered in England and Wales with registered number 05169596 and VAT registered number 223837508. Our registered office is at Crown House, Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended solely for the addressee(s) and may contain confidential information.
If you have received this message in error, please send it back to us, and immediately and permanently delete it.
Do not use, copy or disclose the information contained in this message or in any attachment.
Mobica complies with all requirements of GDPR and other relevant data protection law. You can view our Privacy Policy at https://mobica.com/privacy-policy/

Hi Michal,

That is really strange.

I was not able to reproduce the Driver test failures you saw. They all pass for me, on both X86 and AArch64. I did see some libomptarget failures on AArch64, but I believe those are failing on upstream LLVM 12 as well.

The CMake and ninja incantations I used to configure and build classic-flang-llvm-project were:

cmake -G Ninja
-DCMAKE_BUILD_TYPE=$buildtype
-DCMAKE_INSTALL_PREFIX=$installdir
-DCMAKE_C_COMPILER=gcc
-DCMAKE_CXX_COMPILER=g++
-DLLVM_ENABLE_PROJECTS=“clang;openmp”
-DLLVM_ENABLE_ASSERTIONS=on
-DLLVM_ENABLE_CLASSIC_FLANG=on
-DLLVM_OPTIMIZED_TABLEGEN=on
-DLLVM_TARGETS_TO_BUILD=“X86;AArch64”
…/…/llvm-project/llvm || exit 1
ninja -v check-all install 2>&1

I used GCC 9.3 instead of GCC 10, but I don’t think that matters.

Is anyone else seeing the same failures?

Thanks for checking this, Bryan.

First of all - the flang failure was just a local glitch on my side and was a false alert, sorry about this. So flang repo as such seems safe, all I can see is test failures for clang driver.

I hope the same holds true for the driver failures, however I managed to also reproduce them for clang-12.0.0 and then again more time with gcc-10.1.0. and gcc-9.3.0.
I used the following command for configuration, I hope the configs are the same? I did not build aarch64 as I don’t have the native processor available.

Command I used is the same as yours, I just paste actual values (Release build, etc.), to give all information:

cmake
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr/local
-DCMAKE_C_COMPILER=/usr/bin/gcc-9
-DCMAKE_CXX_COMPILER=/usr/bin/g+±9
-DLLVM_ENABLE_PROJECTS=“clang;openmp”
-DLLVM_ENABLE_ASSERTIONS=on
-DLLVM_ENABLE_CLASSIC_FLANG=ON
-DLLVM_OPTIMIZED_TABLEGEN=on
-DLLVM_TARGETS_TO_BUILD=“X86”
…/llvm

I later also tried to build with ninja. Altogether I tested the following configurations:

  • clang-12 + make (12.0.0)
  • clang-12 + ninja
  • gcc-10 + make (10.1.0)
  • gcc-10 + ninja
  • gcc-9 + ninja ( gcc-9.3.0 to be exact - should be exactly the same setup as yours?)

All of them gave me the same four errors I mentioned above.

Details of my environment setup (Ubuntu-18.04):

  • cmake-3.18.4
  • ninja-1.8.2
  • make-4.1

I am at: commit c1851402a4572a8454fe2255f7ef3bae0516f1d3 (HEAD → release_12x, origin/release_12x).

Can you let me know your tools versions for comparison?

Best regards,

Michał Paszta
Senior Consultant / Senior Group Manager

Mobica Ltd.
Address: Skierniewicka 10a, 01-230 Warszawa

Hi Michal,

That is really strange.

I was not able to reproduce the Driver test failures you saw. They all pass for me, on both X86 and AArch64. I did see some libomptarget failures on AArch64, but I believe those are failing on upstream LLVM 12 as well.

The CMake and ninja incantations I used to configure and build classic-flang-llvm-project were:

cmake -G Ninja
-DCMAKE_BUILD_TYPE=$buildtype
-DCMAKE_INSTALL_PREFIX=$installdir
-DCMAKE_C_COMPILER=gcc
-DCMAKE_CXX_COMPILER=g++
-DLLVM_ENABLE_PROJECTS=“clang;openmp”
-DLLVM_ENABLE_ASSERTIONS=on
-DLLVM_ENABLE_CLASSIC_FLANG=on
-DLLVM_OPTIMIZED_TABLEGEN=on
-DLLVM_TARGETS_TO_BUILD=“X86;AArch64”
…/…/llvm-project/llvm || exit 1
ninja -v check-all install 2>&1

I used GCC 9.3 instead of GCC 10, but I don’t think that matters.

Is anyone else seeing the same failures?


Bryan Chan

Hi Bryan,

Thanks a lot for your efforts with creating this branch. I checked out both your classic-flang-llvm-project/release_12x branch and the code in PR #1010, that you mentioned. They both built fine for me with gcc-10, but I got some errors from the tests I ran. I expect we should also see them in CI, once the release_12x branch is pulled into the origin repo.

Classic-flang-llvm-project errors:

Failed Tests (4):
Clang :: Driver/flang/classic-flang-must-preprocess.F
Clang :: Driver/flang/classic-flang-must-preprocess.F95
Clang :: Driver/flang/classic-flang.f
Clang :: Driver/flang/classic-flang.f95

make install went fine despite the test failures.
With the new llvm-12 I ran the flang project tests and got this error:

Failed Tests (1):
Flang :: f90_correct/lit/array_constructor_nested_implied_do.sh

This comes from a test I just added recently in PR#1004, so perhaps it just needs a slight adjustment to the newer llvm tools. I can check this.

I am more worried about the llvm project failures. We do run clang and llvm tests on every PR, but when pulling the branch as a whole we won’t see them until the first PR to the new branch is submitted for review (just like was the case with release_11x).
Are you able to reproduce these failures on your side?

Best regards,

Michał Paszta
Senior Consultant / Senior Group Manager

Mobica Ltd.
Address: Skierniewicka 10a, 01-230 Warszawa

Hi,

I have created a candidate release_12x branch in classic-flang-llvm-project. It is a rebase of all commits from release_11x since 43ff75f2, minus the DebugInfo additions that already exist in upstream LLVM 12. To build Classic Flang with this branch, we also need some minor changes to Classic Flang itself (see PR #1010). I have tested the build on both X86 and AArch64, and made sure all regression tests pass.

Any help from the community to review and test this branch would be greatly appreciated. Once the final LLVM 12.0.0 is tagged, I will rebase the branch and pull it into classic-flang-llvm-project, if there is no objection from the community.

Thanks,


Bryan Chan


flang-dev mailing list
flang-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev

Mobica is a global software services company, delivering and enabling technologies that transform business outcomes for the leading brands in Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow UK with offices across Europe and the US, our established technical and delivery excellence in high quality software engineering drives success for our multinational customers on every continent, every day.
Find out more at Mobica.com

Mobica Limited is a limited company registered in England and Wales with registered number 05169596 and VAT registered number 223837508. Our registered office is at Crown House, Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended solely for the addressee(s) and may contain confidential information.
If you have received this message in error, please send it back to us, and immediately and permanently delete it.
Do not use, copy or disclose the information contained in this message or in any attachment.
Mobica complies with all requirements of GDPR and other relevant data protection law. You can view our Privacy Policy at https://mobica.com/privacy-policy/

Mobica is a global software services company, delivering and enabling technologies that transform business outcomes for the leading brands in Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow UK with offices across Europe and the US, our established technical and delivery excellence in high quality software engineering drives success for our multinational customers on every continent, every day.
Find out more at Mobica.com

Mobica Limited is a limited company registered in England and Wales with registered number 05169596 and VAT registered number 223837508. Our registered office is at Crown House, Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended solely for the addressee(s) and may contain confidential information.
If you have received this message in error, please send it back to us, and immediately and permanently delete it.
Do not use, copy or disclose the information contained in this message or in any attachment.
Mobica complies with all requirements of GDPR and other relevant data protection law. You can view our Privacy Policy at https://mobica.com/privacy-policy/

Hi Michał,

I rebuilt everything from scratch today on my Ubuntu 20.04.2 machine (x86_64). “ninja check-all” succeeded for me. I also ran those four Driver test cases by hand to confirm that they passed.

The commits I used were:

c1851402a4572a8454fe2255f7ef3bae0516f1d3 classic-flang-llvm-project
01c15cbe7acff988e526de75ad5c61ab09f44f48 flang

The tools I used were:

  • gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
  • GNU Make 4.2.1
  • cmake version 3.16.3
  • ninja 1.10.0

For the complete build script I used, see:

https://github.com/Huawei-PTLab/classic-flang-llvm-project/wiki

Regards,

Hi Bryan,

I checked that I have the exact same issue with release_100 of classic-flang-llvm-project. This issue is local and has nothing to do with your changes.

I took a deeper look to identify the root cause of my issue and it turns out that the RUN calls for flang1 and flang2 expect the tools to be called without the full path, like so:
! CHECK-OBJECT: “flang1”

However on my machine the call has the full path and looks like this: “/usr/local/bin/flang1”.

I managed to fix the issue by letting the checker accept the path if it’s there:
! CHECK-OBJECT: “{{.*}}flang1”

If this makes sense, I will submit a PR with a patch fixing this, later on.

Anyway, from my point of view, the release_12x branch looks safe to be pulled to the origin repo and the changes to flang also all look reasonable and correct.
I am sorry for slowing down the review process with issues which are in fact glitches in my local setup.

Michał Paszta

Hi Michał,

I rebuilt everything from scratch today on my Ubuntu 20.04.2 machine (x86_64). “ninja check-all” succeeded for me. I also ran those four Driver test cases by hand to confirm that they passed.

The commits I used were:

c1851402a4572a8454fe2255f7ef3bae0516f1d3 classic-flang-llvm-project
01c15cbe7acff988e526de75ad5c61ab09f44f48 flang

The tools I used were:

  • gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
  • GNU Make 4.2.1
  • cmake version 3.16.3
  • ninja 1.10.0

For the complete build script I used, see:

https://github.com/Huawei-PTLab/classic-flang-llvm-project/wiki

Regards,


Bryan Chan

Thanks for checking this, Bryan.

First of all - the flang failure was just a local glitch on my side and was a false alert, sorry about this. So flang repo as such seems safe, all I can see is test failures for clang driver.

I hope the same holds true for the driver failures, however I managed to also reproduce them for clang-12.0.0 and then again more time with gcc-10.1.0. and gcc-9.3.0.
I used the following command for configuration, I hope the configs are the same? I did not build aarch64 as I don’t have the native processor available.

Command I used is the same as yours, I just paste actual values (Release build, etc.), to give all information:

cmake
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr/local
-DCMAKE_C_COMPILER=/usr/bin/gcc-9
-DCMAKE_CXX_COMPILER=/usr/bin/g+±9
-DLLVM_ENABLE_PROJECTS=“clang;openmp”
-DLLVM_ENABLE_ASSERTIONS=on
-DLLVM_ENABLE_CLASSIC_FLANG=ON
-DLLVM_OPTIMIZED_TABLEGEN=on
-DLLVM_TARGETS_TO_BUILD=“X86”
…/llvm

I later also tried to build with ninja. Altogether I tested the following configurations:

  • clang-12 + make (12.0.0)
  • clang-12 + ninja
  • gcc-10 + make (10.1.0)
  • gcc-10 + ninja
  • gcc-9 + ninja ( gcc-9.3.0 to be exact - should be exactly the same setup as yours?)

All of them gave me the same four errors I mentioned above.

Details of my environment setup (Ubuntu-18.04):

  • cmake-3.18.4
  • ninja-1.8.2
  • make-4.1

I am at: commit c1851402a4572a8454fe2255f7ef3bae0516f1d3 (HEAD → release_12x, origin/release_12x).

Can you let me know your tools versions for comparison?

Best regards,

Michał Paszta
Senior Consultant / Senior Group Manager

Mobica Ltd.
Address: Skierniewicka 10a, 01-230 Warszawa

Hi Michal,

That is really strange.

I was not able to reproduce the Driver test failures you saw. They all pass for me, on both X86 and AArch64. I did see some libomptarget failures on AArch64, but I believe those are failing on upstream LLVM 12 as well.

The CMake and ninja incantations I used to configure and build classic-flang-llvm-project were:

cmake -G Ninja
-DCMAKE_BUILD_TYPE=$buildtype
-DCMAKE_INSTALL_PREFIX=$installdir
-DCMAKE_C_COMPILER=gcc
-DCMAKE_CXX_COMPILER=g++
-DLLVM_ENABLE_PROJECTS=“clang;openmp”
-DLLVM_ENABLE_ASSERTIONS=on
-DLLVM_ENABLE_CLASSIC_FLANG=on
-DLLVM_OPTIMIZED_TABLEGEN=on
-DLLVM_TARGETS_TO_BUILD=“X86;AArch64”
…/…/llvm-project/llvm || exit 1
ninja -v check-all install 2>&1

I used GCC 9.3 instead of GCC 10, but I don’t think that matters.

Is anyone else seeing the same failures?


Bryan Chan

Hi Bryan,

Thanks a lot for your efforts with creating this branch. I checked out both your classic-flang-llvm-project/release_12x branch and the code in PR #1010, that you mentioned. They both built fine for me with gcc-10, but I got some errors from the tests I ran. I expect we should also see them in CI, once the release_12x branch is pulled into the origin repo.

Classic-flang-llvm-project errors:

Failed Tests (4):
Clang :: Driver/flang/classic-flang-must-preprocess.F
Clang :: Driver/flang/classic-flang-must-preprocess.F95
Clang :: Driver/flang/classic-flang.f
Clang :: Driver/flang/classic-flang.f95

make install went fine despite the test failures.
With the new llvm-12 I ran the flang project tests and got this error:

Failed Tests (1):
Flang :: f90_correct/lit/array_constructor_nested_implied_do.sh

This comes from a test I just added recently in PR#1004, so perhaps it just needs a slight adjustment to the newer llvm tools. I can check this.

I am more worried about the llvm project failures. We do run clang and llvm tests on every PR, but when pulling the branch as a whole we won’t see them until the first PR to the new branch is submitted for review (just like was the case with release_11x).
Are you able to reproduce these failures on your side?

Best regards,

Michał Paszta
Senior Consultant / Senior Group Manager

Mobica Ltd.
Address: Skierniewicka 10a, 01-230 Warszawa

Hi,

I have created a candidate release_12x branch in classic-flang-llvm-project. It is a rebase of all commits from release_11x since 43ff75f2, minus the DebugInfo additions that already exist in upstream LLVM 12. To build Classic Flang with this branch, we also need some minor changes to Classic Flang itself (see PR #1010). I have tested the build on both X86 and AArch64, and made sure all regression tests pass.

Any help from the community to review and test this branch would be greatly appreciated. Once the final LLVM 12.0.0 is tagged, I will rebase the branch and pull it into classic-flang-llvm-project, if there is no objection from the community.

Thanks,


Bryan Chan


flang-dev mailing list
flang-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/flang-dev

Mobica is a global software services company, delivering and enabling technologies that transform business outcomes for the leading brands in Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow UK with offices across Europe and the US, our established technical and delivery excellence in high quality software engineering drives success for our multinational customers on every continent, every day.
Find out more at Mobica.com

Mobica Limited is a limited company registered in England and Wales with registered number 05169596 and VAT registered number 223837508. Our registered office is at Crown House, Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended solely for the addressee(s) and may contain confidential information.
If you have received this message in error, please send it back to us, and immediately and permanently delete it.
Do not use, copy or disclose the information contained in this message or in any attachment.
Mobica complies with all requirements of GDPR and other relevant data protection law. You can view our Privacy Policy at https://mobica.com/privacy-policy/

Mobica is a global software services company, delivering and enabling technologies that transform business outcomes for the leading brands in Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow UK with offices across Europe and the US, our established technical and delivery excellence in high quality software engineering drives success for our multinational customers on every continent, every day.
Find out more at Mobica.com

Mobica Limited is a limited company registered in England and Wales with registered number 05169596 and VAT registered number 223837508. Our registered office is at Crown House, Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended solely for the addressee(s) and may contain confidential information.
If you have received this message in error, please send it back to us, and immediately and permanently delete it.
Do not use, copy or disclose the information contained in this message or in any attachment.
Mobica complies with all requirements of GDPR and other relevant data protection law. You can view our Privacy Policy at https://mobica.com/privacy-policy/

Mobica is a global software services company, delivering and enabling technologies that transform business outcomes for the leading brands in Automotive, Silicon, FinTech, Media and Telecoms. Headquartered in Wilmslow UK with offices across Europe and the US, our established technical and delivery excellence in high quality software engineering drives success for our multinational customers on every continent, every day.
Find out more at Mobica.com

Mobica Limited is a limited company registered in England and Wales with registered number 05169596 and VAT registered number 223837508. Our registered office is at Crown House, Manchester Road, Wilmslow, Cheshire, SK9 1BH, UK.
This message is intended solely for the addressee(s) and may contain confidential information.
If you have received this message in error, please send it back to us, and immediately and permanently delete it.
Do not use, copy or disclose the information contained in this message or in any attachment.
Mobica complies with all requirements of GDPR and other relevant data protection law. You can view our Privacy Policy at https://mobica.com/privacy-policy/

Thanks Michał for getting to the bottom of this!