Building LLVM out of per subproject tar balls still is not working

LLVM is really big project and packaging its sub projects as separated packages is really useful.
Issue is that it still does not work like it should be.

Bolt build fails

Several other projects are not fully working (In this case I’ve attached even proposed patch but looks like one is interested)

libcxx, libcxxabi circular build dependencies

clang issue (this one is solved by patch which is proposed by fedora clang maintainer)

Is it any intention to bring that to working state?

This comes up from time to time but there is really low interest from the community to make standalone builds work. I think it will only be rectified if someone steps up and works on it.

I think the only reason we still publish the separate tarballs is because some distributions need it. But I think we maybe should rethink this considering how busted standalone builds are.

If no one from maintainers cares and it does not work why still those dist tar balls are provided? :thinking:

There are many discussions about this in various threads:

And

Those might give you an insight into why they are still around.

Agree.

In general, My feeling is that Linux distribution packagers (at least Gentoo (@mgorny) and RedHat/Fedora (@tstellar)) care about stand-alone build support, but most personal and corporate contributors use the whole repository.

Almost all upstream and downstream build bots use the whole repository - this gives the community coverage for many configurations (there are numerous CMake variables to tune for a build). But the standalone build has only testing for very specific configurations, just the few that some distributions use. So there are likely many rough edges.

Up to now none of the replies changes any state of broken per subproject source dist tar balls. :frowning_face:
Is it possible at least commit patch which I’v proposed?

If you submit patches via Phabricator, I would be surprised if they where denied. You might have a problem finding a reviewer since no-one really cares about these things, but offering the patches always helps push things forward.

The per sub-project tar balls are really meant to be used the same way one would use sparse checkouts from the git repository. There are still dependencies between sub-projects that make it impossible to build some of the sub-projects with a single tarball. For llvm, clang, and lld, the required sub-directories are documented here.

@kloczek I think the best way to approach this problem would be to document which dependencies are currently needed for the various tarballs you are interested in. Once we have this, we can look at how to prune the dependencies and fix the build issues if necessary.

Also, If you have some outstanding patches, you can add me as a reviewer and/or ping them, so they show up in my email.

I cannot agree with above.
LLVM as the project is HUGE.
In case if it will be something to fix by apply single small patch on clang code it wille be possible to rebuild only clang without wasting time on rebuilding everything.
Issue is only with libcxx and libcxxabi which requires each other on build stage so it makes sense to merge them. Such project even combined will be still one of the smallest chunks of the LLVM.

Second thing is that subproject like bolt is not ready to be build out of per subproject tar ball is not working at all which is possible to see in Fedora rpm spec file
https://src.fedoraproject.org/rpms/llvm-bolt/blob/rawhide/f/llvm-bolt.spec

The fact still stands: there is very little interest from the maintainers to have this feature. Someone would have to step up and work on it. Otherwise I think we should remove it, since it’s currently broken.

I don’t follow the argumentation here. Fedora apparently builds bolt based on a tarball (Tree - rpms/llvm-bolt - src.fedoraproject.org):

Source0: https://github.com/llvm/llvm-project/releases/download/llvmorg-%{maj_ver}.%{min_ver}.%{patch_ver}%{?rc_ver:-rc%{rc_ver}}/%{bolt_srcdir}.tar.xz

What is not working here? I mean, sure you depend on LLVM and other stuff:

BuildRequires: llvm-devel = %{version}
BuildRequires: llvm-test = %{version}
BuildRequires: python3-lit

But that’s intended.

Maybe I’m just not getting the point of your post.

@tobiashieta what exactly is broken? Some projects refuse to work on standalone builds and others don’t support them at all or have abandoned them but this is not true for llvm, clang, lld, openmp, compiler-rt and so on. They still work “just fine” from a linux distribution perspective. The amount of downstream patches mostly doesn’t focus on standalone builds but on other stuff, at least for Fedora.

PR: [BOLT] Enable standalone build by pca006132 · Pull Request #87196 · llvm/llvm-project · GitHub