Have you been sleeping poorly worried about the Clang 3.5 release? Well, this may help! The plan right now is to start testing in July with August as the target release month. There isn’t a schedule yet, of course. But it should be a goodly amount of time for all y’all to prepare for the release process.
If you have any questions, please let me know. If you’d like to volunteer to be a tester, also let me know. 
Share and enjoy!
-bw
Happy to test on Ubuntu 14.04 x86_64.
Ben
I'm happy to do Windows binaries.
Thanks,
Hans
Count me in for Fedora and OpenSUSE. Might give Windows a spin as well.
As always, I’m happy to do the OS X stuff.
Cheers,
Sebastian
Count me in too ! Gentoo & Ubuntu 12.04
Cheers,
In article <4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com>,
Bill Wendling <isanbard@gmail.com> writes:
Have you been sleeping poorly worried about the Clang 3.5 release? Well,
this may help! The plan right now is to start testing in July with August
as the target release month. There isn't a schedule yet, of course. But it
should be a goodly amount of time for all y'all to prepare for the release
process.If you have any questions, please let me know. If you'd like to volunteer
to be a tester, also let me know.
I have some packaging changes I'd like to see in place for the 3.5
packaging. The changes include some more utilities in the package for
refactoring tool developers and there is an issue with the way the
Windows packages are being built that causes them to omit many of the
things in the unix packages. Again, this is to make it easier to
write refactoring tools.
So far the idea with the Windows installer has been to provide a toolchain rather than a complete SDK, but it'd be nice to see if we can head in that direction, perhaps with a separate SDK package.
CC'ing in Hans who has opinions on this.
Which files did you want to include specifically?
Alp.
Linux on ARM tests here.
Initially I didn't want to do this because it bloats the installation. The
package itself is compressed, so a lot of the code duplication is mitigated.
Normally people solve this with shared libraries, and I assumed that's what
we did on Linux, where we have a shared library build. Turns out I was
wrong, our pre-built binaries are totally static. So maybe this doesn't
matter as much as I thought it did.
Relatedly, I think at some point we should add LLVM_EXPORT annotations to
APIs in Support/ADT, IR, Target, etc, so that we can reduce our .dynsym
count and actually support an LLVM.dll on Windows. However, I know there
is *strong* resistance in some camps due to the burden this would impose.
I know Richard has been working on using the tooling stuff on Windows,
but I don't really know how it works - is there an example somewhere
that shows what would need to be included in the installer?
If we want to include it in the 3.5 release, we should get it into the
snapshot builds first.
Thanks,
Hans
In article <53765DB4.8040009@nuanti.com>,
Alp Toker <alp@nuanti.com> writes:
> I have some packaging changes I'd like to see in place for the 3.5
> packaging. [...]So far the idea with the Windows installer has been to provide a
toolchain rather than a complete SDK, but it'd be nice to see if we can
head in that direction, perhaps with a separate SDK package.
Yes, this was stated before.
However, it makes no sense to me.
Why are we denying Windows users of the clang package libraries and
headers that we supply to the users on other platforms?
I see no reason for the platform disparity.
CC'ing in Hans who has opinions on this.
This message didn't have any cc: header on it when I got it... FYI.
Which files did you want to include specifically?
See <http://llvm.org/bugs/show_bug.cgi?id=19223>.
Libraries and headers for using the refactoring tool library code are
present in the unix distributions but not present in the Windows
distributions.
For this particular situation, the problem isn't that the build needs
a patch, it's that you need to stop setting
LLVM_INSTALL_TOOLCHAIN_ONLY=ON on the machine where you package up the
Windows package.
Off the top of my head I can think of two clang tools that would be
useful to developers of refactoring tools: clang-query and FileCheck.
clang-query wasn't in the package on any platform in 3.4, but the last
time I looked at 3.5 trunk, this was now packaged. In the release 3.4
branch, clang-query also required libedit in order to be built at all
(which was really only fancy command editing, not required for the core
functionality). This has also been fixed on 3.5 trunk. I had a patch
for the version on 3.4 and when I went to see about submitting it to 3.5,
someone had already beaten me to it :).
FileCheck is useful in order to be able to run automated acceptance
tests of your command-line refactoring tool against source files.
Currently this file is not included in the package either, although it's
less important than clang-query.
When I was reviewing all the utilities that were being built and weren't
being included in any package, I believe I came across a few more that
would be useful, but clang-query and FileCheck are probably the most
important two.
OK, maybe we should try shipping without LLVM_INSTALL_TOOLCHAIN_ONLY.
It grows the installer from 25 MB to 115 MB, but Reid pointed out that
the release on Linux is actually even larger, so maybe it's ok.
If we think this is what we want to do, I can build next week's
snapshot this way, and maybe Richard can try it out and see if
building a tool with that works?
Thanks,
Hans
In article <CAB8jPhdE5mE8v=OVw3A4N2cxLNNhZTdisbTnaMUkxD+erizs+Q@mail.gmail.com>,
Hans Wennborg <hans@chromium.org> writes:
OK, maybe we should try shipping without LLVM_INSTALL_TOOLCHAIN_ONLY.
It grows the installer from 25 MB to 115 MB, but Reid pointed out that
the release on Linux is actually even larger, so maybe it's ok.
I bought a copy of "Mastering CMake" and read it so I could understand
clang/llvm's build. I understand it enough that I think I could try
creating an "SDK" installer that would be for developers using either
libclang or the C++ API.
If we think this is what we want to do, I can build next week's
snapshot this way, and maybe Richard can try it out and see if
building a tool with that works?
I'm very willing to test that.
In article <4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com
<mailto:4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com>>,
Bill Wendling <isanbard@gmail.com
<mailto:isanbard@gmail.com>> writes:Have you been sleeping poorly worried about the Clang 3.5
release? Well,
this may help! The plan right now is to start testing in
July with August
as the target release month. There isn't a schedule yet,
of course. But it
should be a goodly amount of time for all y'all to prepare
for the release
process.If you have any questions, please let me know. If you'd
like to volunteer
to be a tester, also let me know.I have some packaging changes I'd like to see in place for the 3.5
packaging. The changes include some more utilities in the
package for
refactoring tool developers and there is an issue with the way the
Windows packages are being built that causes them to omit many
of the
things in the unix packages. Again, this is to make it easier to
write refactoring tools.So far the idea with the Windows installer has been to provide a
toolchain rather than a complete SDK, but it'd be nice to see if
we can head in that direction, perhaps with a separate SDK package.CC'ing in Hans who has opinions on this.
Which files did you want to include specifically?
Initially I didn't want to do this because it bloats the installation. The package itself is compressed, so a lot of the code duplication is mitigated.
Normally people solve this with shared libraries, and I assumed that's what we did on Linux, where we have a shared library build. Turns out I was wrong, our pre-built binaries are totally static. So maybe this doesn't matter as much as I thought it did.
Relatedly, I think at some point we should add LLVM_EXPORT annotations to APIs in Support/ADT, IR, Target, etc,
I agree that it would be absolutely fantastic to support shared library / dll builds in the configuration you describe and think we should work towards that.
I hope however that we _never_ attempt to do it with annotations like LLVM_EXPORT in the source core, nor with export lists. The reason:
1) They're a complete pain to maintain for developers not working on Windows.
2) Even on Windows, there are at least two possible dynamic linking configurations:
(a) several modular DLLs
(b) LLVM.dll + clang.dll
(c) LLVMClang.dll
Annotations could only ever support one of the above given that what's exported and what's imported is different for each.
so that we can reduce our .dynsym count and actually support an LLVM.dll on Windows.
So, it strikes me that this problem has great parallels with LTO, specifically the task of calculating the minimal set of symbols and interdependencies between modules (in this case shared libraries and executables).
Perhaps we can treat this as a technology problem and hook into LTO or introduce some similar analysis to calculate shared library import / exports without having to pepper them throughout the source code.
Rafael and I were discussing something like in person recently. Rafael, can you think of a way the LTO machinery could assist in determining DLL imports and exports at build time?
However, I know there is *strong* resistance in some camps due to the burden this would impose.
That resistance is valid. Annotations have been a total mess in the projects I've seen them applied. Apart from LLVM I'm sure other projects like WebKit / Chromium wanting to support DLL builds would benefit from any solution we come up with.
Alp.
In article <4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com
<mailto:4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com>>,
Bill Wendling <isanbard@gmail.com<mailto:isanbard@gmail.com>> writes:
Have you been sleeping poorly worried about the Clang 3.5
release? Well,
this may help! The plan right now is to start testing in
July with August
as the target release month. There isn't a schedule yet,
of course. But it
should be a goodly amount of time for all y'all to prepare
for the release
process.If you have any questions, please let me know. If you'd
like to volunteer
to be a tester, also let me know.I have some packaging changes I'd like to see in place for the 3.5
packaging. The changes include some more utilities in the
package for
refactoring tool developers and there is an issue with the way the
Windows packages are being built that causes them to omit many
of the
things in the unix packages. Again, this is to make it easier to
write refactoring tools.So far the idea with the Windows installer has been to provide a
toolchain rather than a complete SDK, but it'd be nice to see if
we can head in that direction, perhaps with a separate SDK package.CC'ing in Hans who has opinions on this.
Which files did you want to include specifically?
Initially I didn't want to do this because it bloats the installation.
The package itself is compressed, so a lot of the code duplication is
mitigated.Normally people solve this with shared libraries, and I assumed that's
what we did on Linux, where we have a shared library build. Turns out I
was wrong, our pre-built binaries are totally static. So maybe this
doesn't matter as much as I thought it did.Relatedly, I think at some point we should add LLVM_EXPORT annotations to
APIs in Support/ADT, IR, Target, etc,I agree that it would be absolutely fantastic to support shared library /
dll builds in the configuration you describe and think we should work
towards that.I hope however that we _never_ attempt to do it with annotations like
LLVM_EXPORT in the source core, nor with export lists. The reason:1) They're a complete pain to maintain for developers not working on
Windows.
I don't have a dog in this race either way, but I should point out that if
you build with -fvisibility=hidden and let your annotation expand
to __attribute__((visibility("default"))) while compiling your library and
to nothing for clients of your library, non-Windows shared builds will work
exactly when the the Windows shared build works. (This is how the Chromium
components build works, and it works fairly well there as far as I can
tell.)
In article
<4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com
<mailto:4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com>
<mailto:4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com
<mailto:4FBFE344-19A4-4A9A-9C93-AB9A4554D6CE@gmail.com>>>,
Bill Wendling <isanbard@gmail.com
<mailto:isanbard@gmail.com><mailto:isanbard@gmail.com
<mailto:isanbard@gmail.com>>> writes:Have you been sleeping poorly worried about the
Clang 3.5
release? Well,
this may help! The plan right now is to start
testing in
July with August
as the target release month. There isn't a
schedule yet,
of course. But it
should be a goodly amount of time for all y'all to
prepare
for the release
process.If you have any questions, please let me know. If
you'd
like to volunteer
to be a tester, also let me know.I have some packaging changes I'd like to see in place
for the 3.5
packaging. The changes include some more utilities in the
package for
refactoring tool developers and there is an issue with
the way the
Windows packages are being built that causes them to
omit many
of the
things in the unix packages. Again, this is to make
it easier to
write refactoring tools.So far the idea with the Windows installer has been to
provide a
toolchain rather than a complete SDK, but it'd be nice to
see if
we can head in that direction, perhaps with a separate SDK
package.CC'ing in Hans who has opinions on this.
Which files did you want to include specifically?
Initially I didn't want to do this because it bloats the
installation. The package itself is compressed, so a lot of
the code duplication is mitigated.Normally people solve this with shared libraries, and I
assumed that's what we did on Linux, where we have a shared
library build. Turns out I was wrong, our pre-built binaries
are totally static. So maybe this doesn't matter as much as I
thought it did.Relatedly, I think at some point we should add LLVM_EXPORT
annotations to APIs in Support/ADT, IR, Target, etc,I agree that it would be absolutely fantastic to support shared
library / dll builds in the configuration you describe and think
we should work towards that.I hope however that we _never_ attempt to do it with annotations
like LLVM_EXPORT in the source core, nor with export lists. The
reason:1) They're a complete pain to maintain for developers not working
on Windows.I don't have a dog in this race either way, but I should point out that if you build with -fvisibility=hidden and let your annotation expand to __attribute__((visibility("default"))) while compiling your library and to nothing for clients of your library, non-Windows shared builds
That implies all LLVM developers would have to maintain a shared build *precisely* matching whatever DLL layout is chosen for Windows, using the same feature enable flags and building the same set of LLVM SVN modules.
Even assuming all LLVM developers switch to the shared library build, anyone working on LLVM core without clang would be left out in the cold and liable to break the build with every commit. The whole idea relies on a chain of flaky assumptions and even then it's a pain to maintain.
will work exactly when the the Windows shared build works. (This is how the Chromium components build works, and it works fairly well there as far as I can tell.)
Nico, that sounds like confirmation bias. I can say from first hand experience that the Chromium import/exports don't serve projects outside of Google because they're hard-coded to match the distribution layout you guys are using for the Google Chrome product.
Exclude a single module, try to split out a component for Debian/Fedora packages etc. and the annotations become entirely unusable. This won't be acceptable for LLVM.
Alp.
Rafael and I were discussing something like in person recently. Rafael, can
you think of a way the LTO machinery could assist in determining DLL imports
and exports at build time?
I don't think it is LTO specific. We could write a mode for LLD where
it figures out exports automatically on COFF, but we should probably
not depend on that for building a release.
I think I am with Nico on this one. Building with -fvisibility-hidden
would probably be a good thing for LLVM anyway. Once we have that then
hopefully windows dll builds would not be too hard.
Cheers,
Rafael
Rafael and I were discussing something like in person recently. Rafael, can
you think of a way the LTO machinery could assist in determining DLL imports
and exports at build time?I don't think it is LTO specific. We could write a mode for LLD where
it figures out exports automatically on COFF, but we should probably
not depend on that for building a release.I think I am with Nico on this one. Building with -fvisibility-hidden
would probably be a good thing for LLVM anyway. Once we have that then
hopefully windows dll builds would not be too hard.
I don't think maintaining import/export annotations in the source code is viable in LLVM.
We'd be looking at hundreds of macros like EXPORT_IF_HAVE_CLANG and IMPORT_IF_MODULAR_BUILD, EXPORT_IF_NOT_MONOLITHIC_LLVMCLANG attached to functions and classes throughout LLVM. Yuck 
Excessive exports would prevent optimisations and inlining without any clear way to prune them as the code changes. LTO would have to export all these functions just in case something might be using them.
The only way to detect when new exports are needed would be to observe build bot breakage and add them incrementally until the bots turn green. Or to keep a shared build matching the DLL configuration.
It's really burdensome to have to modify the original *definition* every time you want to use a class or function. You'll no longer be able to develop and test a local clang branch against upstream LLVM, which will discourage people from developing against ToT.
And finally, the risk of breaking the build will be high enough to tip the balance in favour of copy-and-pasting the necessary code from LLVM core into clang in order to use it. It's not like we don't already have a copy-and-paste problem and this would legitimise the practice.
TLDR; We _really_ want to calculate these dependencies at build time with LLVMBuild instead of annotating the source.
Alp.
I don't think maintaining import/export annotations in the source code is
viable in LLVM.We'd be looking at hundreds of macros like EXPORT_IF_HAVE_CLANG and
IMPORT_IF_MODULAR_BUILD, EXPORT_IF_NOT_MONOLITHIC_LLVMCLANG attached to
functions and classes throughout LLVM. Yuck
Well, we can always start with the most conservative case, no? Just a
LLVM_EXPORT that gets added if the symbol is needed in *any*
configuration. That would be better than what we have now for ELF and
would make it possible to get dll builds on windows.
Excessive exports would prevent optimisations and inlining without any clear
way to prune them as the code changes. LTO would have to export all these
functions just in case something might be using them.
That is the current situation on ELF/MachO
The only way to detect when new exports are needed would be to observe build
bot breakage and add them incrementally until the bots turn green. Or to
keep a shared build matching the DLL configuration.It's really burdensome to have to modify the original *definition* every
time you want to use a class or function. You'll no longer be able to
develop and test a local clang branch against upstream LLVM, which will
discourage people from developing against ToT.And finally, the risk of breaking the build will be high enough to tip the
balance in favour of copy-and-pasting the necessary code from LLVM core into
clang in order to use it. It's not like we don't already have a
copy-and-paste problem and this would legitimise the practice.TLDR; We _really_ want to calculate these dependencies at build time with
LLVMBuild instead of annotating the source.
Is there a convenient way to feed this to link.exe?
Cheers,
Rafael