Build_llvm_release.bat script options

Few years ago @rgal noted that lld-link was not used in stage1, despite building it in stage0.
https://lists.llvm.org/pipermail/llvm-dev/2019-August/134393.html

At Sony we have tried the build_llvm_package.bat script and these are some points that may be we could consider:

  • As the script is part of the LLVM tree, there are users that can use it for their internal releases and benefit from features such as: create binary package, run the tests, etc.
  • Is is true that executing the whole script takes quite a long time. May be add some command line options to select the desired tasks.

These optional tasks can be:

  • Specify output directory.
  • Build 32 and/or 64 binaries.
  • Run the tests
  • Package the binaries.
  • Use lld-link in stage1

We have most of those changes implemented and if the community is interested we can submit a patch.

Thanks
Carlos

Sounds good to me. As long as the complexity of the script doesn’t increase too much, I think all those options sound good.

Use lld-link in stage1

We should probably just turn this on always.

We should also consider using thinlto and pgo, although it will increase both the complexity and run time of the script.

I do feel like ThinLTO is something we should turn on for all our release builds. Since we already multi-stage build the release builds it shouldn’t be to much of a hassle to get this working.

PGO is another beast. I have long thought it would be good to have a performance toolchain only set of packages published considering how much performance the current binaries leave on the table.

Anyway - I am happy to see improvements to the windows script as well and I am happy to help review any patches against it.

I will put a patch for review.
The only “complex” part is a small function doing the command line parsing :slight_smile:

@hansw2000, looking at the current script:

curl -L https://github.com/llvm/llvm-project/archive/%revision%.zip -o src.zip || exit /b

With revision being something like ''14.0.4", you get archive/llvmorg-14.0.4.zip.

In our case, we have been using:

curl -L https://github.com/llvm/llvm-project/archive/refs/heads/main.zip -o src.zip || exit /b

If the script is intended to be used as a general build/packaging, may be it should in some way to allow the seleccion of the source (current head or previous).

The script is mainly intended to build releases, but having command-line options to set the revision/version/build dir would be reasonable.

I have created https://reviews.llvm.org/D127938 with the following changes:

  • Add command line options to select specific tasks:
    • Specify output directory.
    • Build 32 and/or 64 binaries.
    • Run the tests
    • Package the binaries.
  • In stage-1 use lld-link.
  • Split the build/test/package into separate functions.
  • Added some trace statements.

There are more changes that can be done to the script. The 32/64 steps are basically the same. Basically their differences are: Python environment variables and output directories. I woud suggest to refactor them in a different patch.