Automated Github Actions for release testing

Hi,

I just wanted to share what I have setup to automatically help me build/test/upload new versions of LLVM via Github actions.

Github have the possibility for your to add your own runners, so I have added my Mac Mini to my own account as a runner. Then I have a simple workflow that looks like this:

name: LLVM Release test
on:
  workflow_dispatch:
    inputs:
      tag:
        description: "LLVM tag"
        required: true
jobs:
  BuildAndTestMacos:
    runs-on: [self-hosted, macOS, X64]
    steps:
      - name: Check out code
        uses: actions/checkout@v2
      - name: Build
        run: ./build_release.py ${{ github.event.inputs.tag }}
      - name: move
        run: mv build/${{ github.event.inputs.tag }}/*/clang+llvm*.tar.xz .
      - name: Archive LLVM
        uses: actions/upload-artifact@v3
        with:
          path: clang+llvm-*x86_64*.tar.xz
      - name: Archive logs
        uses: actions/upload-artifact@v3
        with:
          path: build/${{ github.event.inputs.tag }}/*/logs/*

This sets some variables and then runs test-release script. The output is then stored as a artifact. I can now use these binaries to test some stuff and inspect the test output logs.

When I am happy with it I have a second workflow that outputs all the info I need for the forum post and uploads the binaries to the release:

name: LLVM upload release
on:
  workflow_dispatch:
    inputs:
      run_number:
        description: Build Run
        required: true
      tag:
        description: LLVM Tag
        required: true

jobs:
  UploadRelease:
    runs-on: [ubuntu-latest]
    steps:
      - name: Download Artifact
        uses: dawidd6/action-download-artifact@v2
        with:
          github_token: ${{ secrets.GH_TOKEN }}
          workflow: github-llvm-release.yml
          run_number: ${{ github.event.inputs.run_number }}
      - name: shasum
        run: sha256sum artifact/*.tar.xz
      - name: Test Failures
        run: "grep ^FAIL: artifact/*/logs/testing.*.log"
      - name: Upload artifact
        uses: softprops/action-gh-release@v1
        with:
          files: |
            artifact/clang+llvm*.tar.xz
          tag_name: llvmorg-${{ github.event.inputs.tag }}
          repository: llvm/llvm-project
          token: ${{ secrets.GH_TOKEN }}

With this setup and when @tstellar announces a new version I can just run gh workflow run in the repo and it will prompt me for the tag and run the rest.

I have kept this repository private since I don’t want others to figure out how to push releases etc - but if someone wants to access the code and expand on it let me know and I can add you.

Another option that we might want to consider is that we automate all the building by adding peoples build servers to a workflow. Let me know if there is any interest in this.

Thanks,
Tobias

This looks great. Thanks for working on it. One thing I wanted to point out is that there is a python script in tree at llvm/utils/release/github-upload-release.py that can upload release artifacts, so it might be best to use that.

I think this would be great, and long-term I think this is what we should be doing. It would also be nice if we could use the Github hosted runners for this, but they don’t currently have enough disk space for the release script.