Hi LLVM devs,
As you are already aware, performing a clean build of LLVM requires considerable computing resources. This presents a barrier to entry for people who do not have access to large computers. Since the buildbots already regularly compile the LLVM codebase, making tarballs of their build directories available on a public facing server will dramatically reduce the barrier to entry. Is this something that the community is willing to consider?
Perhaps these pages may help.
Ah, apologies, I think I wasn’t clear. I am talking about making object files available in this manner so that people can download them and compile just their changes without having to compile all of LLVM, thus reducing the barrier to entry. As far as I am aware, the releases on Github do not contain object files.
That’s an interesting idea. There are several issues to consider:
One bit of warning, build trees tend to have relative paths baked in. From experience, trying to copy an llvm build tree from one location to another tends to not work well.
The best workaround I’ve seen for this type of thing is to use docker (or your container format of choice) to construct a “build image” with the full source and build trees, then update the source and trigger an (incremental) build.
After that’s out of the way, you run into two major problems quickly.
- Such build images tend to be gigantic. (50+gb) Bandwidth costs, storage costs, and download times add up quickly.
- Their value as incremental build sources tend to age very quickly. If you look at the commits to llvm, core headers get changed shocking often. As such, if you’re trying to follow ToT you quickly end up doing what is effectively a clean build anyways. The only workflow which “somewhat works” is to develop against some recent snapshot, then rebase only at the very last and pay the cost of a full build.
I’ve played enough with ideas like this in the past to recommend you not go down this path.
An alternate approach I recommend - and use personally - is to do all builds on an AWS instance. With some basic scripting (here’s mine), you can do fast builds for a couple of dollars a day. I’ve been working this was for about 6 months, and have found it dramatically easier than all the options I played with before.
I run the release tests in an Ubuntu VM that I prepare by building and installing the most recent clang-llvm version. I also install a fair number of Ubuntu distribution packages that seemed to be required for building all of the LLVM distribution from when I was looking at that over a year back. The used portion of the VM after the clang-llvm build, those two projects only, that contains the object files is close to 23GB.
Seems like an easy way to put this together would be to tar the VM and provide it for download. 23GB is certainly large but some popular video games obtained by download are larger.
Decide what you want and I can build the VM with the clang projects and upload that.