I would like to start using GitHub Actions[1] for CI testing on the release/*
branches. As far as I know we don't have any buildbots listening to the
release branches, and I think GitHub Actions are a good way for us to quickly
bring-up some CI jobs there.
My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
but in order to get these running in the main repository, I would need to:
1. Create a new repository in the LLVM organization called 'actions' for storing some custom
builds steps for our CI jobs (see [4]).
2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
branch.
In the future, I would also like to add buil and tests jobs for other sub-projects
once I am able to get those working.
In addition to being used for post-commit testing, having these CI definitions in the
main tree will make it easier for me (or anyone) to do pre-commit testing for the
release branch in a personal fork. It will also allow me to experiment with some new
workflows to help make managing the releases much easier.
I think this will be a good way to test Actions in a low traffic environment to
see if they are something we would want to use for CI on the master branch.
Given that we are close to the end of the 9.0.1 cycle, unless there are any
strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
usefulness. Let me know what you think.
Although I’ve not had any Github Action experience, and despite being one who has raised objections to other parts of adopting github features, this sounds like a sensible idea. It seems odd to me that we don’t already have some form of CI for our releases, so anything that improves on that is great.
This sounds very interesting! +1 from me.
What does this imply for the release testers?
Would it be possible / desirable for us to add self-hosted runners for
other architectures? I'm assuming GitHub only provides x86, right? I
totally missed any details about that in their docs, but it seems to
be implied here [1]. I'm not saying we should go all-possible-arches
from the start, we can definitely try it out only for x86 this time
around if you think that would be the best approach.
That sounds really useful. Would it be possible to include LLDB as
well? We have a subset of tests (unit & lit) that can be run without
Python/SWIG by passing LLDB_DISABLE_PYTHON=ON to CMake.
That sounds really useful. Would it be possible to include LLDB as
well? We have a subset of tests (unit & lit) that can be run without
Python/SWIG by passing LLDB_DISABLE_PYTHON=ON to CMake.
I can try to add an lldb builder. Would you be able to provide me
with the cmake arguments plus a check target where all tests are expected
to pass?
I would like to start using GitHub Actions[1] for CI testing on the release/*
branches. As far as I know we don't have any buildbots listening to the
release branches, and I think GitHub Actions are a good way for us to quickly
bring-up some CI jobs there.
My proposal is to start by adding two post-commit CI jobs to the release/9.x branch.
One for building and testing (ninja checka-all) llvm/clang/lld on Linux,
Windows, and Mac, and another for detecting ABI changes since the 9.0.0 release.
I have already implemented these two CI jobs in my llvm-project fork on GitHub[2][3],
but in order to get these running in the main repository, I would need to:
1. Create a new repository in the LLVM organization called 'actions' for storing some custom
builds steps for our CI jobs (see [4]).
2. Commit yaml CI definitions to the .github/workflows directory in the release/9.x
branch.
In the future, I would also like to add buil and tests jobs for other sub-projects
once I am able to get those working.
In addition to being used for post-commit testing, having these CI definitions in the
main tree will make it easier for me (or anyone) to do pre-commit testing for the
release branch in a personal fork. It will also allow me to experiment with some new
workflows to help make managing the releases much easier.
I think this will be a good way to test Actions in a low traffic environment to
see if they are something we would want to use for CI on the master branch.
Given that we are close to the end of the 9.0.1 cycle, unless there are any
strong objections, I would like to get this enabled by Mon Nov 18, to maximize its
usefulness. Let me know what you think.
The CI tests are running now on the stable branch. Here is the
first run [1]. Passing commits[2] now have a green check mark in
the web UI that you can click on to view the results.
These tests have already caught 1 bug that did not appear in my local
builds, so they are proving to be very useful.
Now that this is in place, I'm going to start looking into some more automation
around backporting fixes to the stable branch. I think we could do something like
filing backport requests as a GitHub issues and then have that issue automatically
trigger a pull request and do some pre-commit testing. I'm still trying to learn
what is technically possible with GitHub and if anyone has any ideas or suggestions,
let me know. I'm hoping to have something in place for the 10.0.1 release cycle.
This sounds very interesting! +1 from me.
What does this imply for the release testers?
Would it be possible / desirable for us to add self-hosted runners for
other architectures? I'm assuming GitHub only provides x86, right? I
totally missed any details about that in their docs, but it seems to
be implied here [1]. I'm not saying we should go all-possible-arches
from the start, we can definitely try it out only for x86 this time
around if you think that would be the best approach.
Nothing changes for release testers at this point. The main goal here is
to get some post-commit testing so we can catch issues in the branch early.
Longer term I think have self-hosted runners would be great, because it would
allow us to fully automate the testing and uploading we do when a release gets
tagged.
You are correct that GitHub only provides x86 machines, however, they don't
have enough disk space (14GB) to be able to run the test-release.sh script, so
adding x86 self-hosted builders with more disk space would still be very useful.
In addition to being used for post-commit testing, having these CI definitions in the
main tree will make it easier for me (or anyone) to do pre-commit testing for the
release branch in a personal fork.
I find this part especially useful. I’m using these actions on a personal branch to validate patches on Linux, OSX and Windows, which gives me more confidence on its portability.
