I'd like to propose the contribution of a Go frontend subproject to the LLVM
project, based on the existing llgo project at https://github.com/go-llvm/llgo .
As with the previous contribution of the Go bindings, I have obtained
permission from all llgo contributors whose code is part of this contribution,
to contribute their changes to the LLVM project and relicense their changes
under the LLVM license. I am also willing to be the code owner for the
llgo subproject.
The frontend would live in the LLVM svn repository and if checked out into
tools/llgo would build as part of the regular LLVM build (with CMake only). We
would be keeping llgo compatible with top-of-tree LLVM, although I imagine
this would be less burdensome than the other subprojects as llgo is written
in Go and depends on the Go bindings previously contributed to LLVM.
llgo depends on certain third-party components, namely a copy of the Go
standard library (libgo), a Go program analysis library (go.tools) and two
library dependencies of the standard library (libbacktrace and libffi). These
would be mirrored into the llgo repository under a third_party directory. They
would retain their original licenses, which are BSD and GPLv3 with Runtime
Library Exception (the latter only applies to a handful of header files;
eventually we would seek to replace these).
with the initial code contribution. The mirrored third-party sources will be
added later, as the diff would be too large to review. If there is consensus
in the community, the next step I propose to take is to create the subproject
in svn and check in the initial version of the code.
Any comments on this contribution are appreciated.
I'd like to propose the contribution of a Go frontend subproject to the LLVM
project, based on the existing llgo project at https://github.com/go-llvm/llgo .
As with the previous contribution of the Go bindings, I have obtained
permission from all llgo contributors whose code is part of this contribution,
to contribute their changes to the LLVM project and relicense their changes
under the LLVM license. I am also willing to be the code owner for the
llgo subproject.
Fantastic. As we discussed at the LLVM Developer Meeting, I’m in favor of this. However, new subprojects need to be approved by the LLVM Foundation board of directors. Chandler can help orchestrate this.
llgo depends on certain third-party components, namely a copy of the Go
standard library (libgo), a Go program analysis library (go.tools) and two
library dependencies of the standard library (libbacktrace and libffi). These
would be mirrored into the llgo repository under a third_party directory. They
would retain their original licenses, which are BSD and GPLv3 with Runtime
Library Exception (the latter only applies to a handful of header files;
eventually we would seek to replace these).
It is up to the board, but MHO is that the GPLv3 pieces should be rewritten before we can accept the code. That way we have purity in the full commit history of the subproject repository (i.e. someone wouldn’t accidentally get GPL3 code by checking out the wrong version number or because they use git).
For the other incorporated dependencies, just make sure to list them (and their licenses) in the top level LICENSE.TXT file.
A quick browse through the contributors on https://github.com/go-llvm/llgo/graphs/contributors and some searches in my mail suggests that none of the primary contributors are part of the LLVM community. How are they going to continue developing llgo once it is the LLVM repo?
Sorry if this is a dumb question, but what is the advantage of having this in-tree as an LLVM project? I typically associate the following advantages to having something in-tree:
You can get the awesome code review of the LLVM community.
If people are changing the API, they have to fix in-tree stuff, so you are shielded from ongoing maintenance burden.
Since the code is in go, you don’t really get 1; the community speaks C++.
IIRC the bindings are based on the C API, so they are stable and so 2 doesn’t apply.
What are the advantages to having it checked in to the LLVM project?
I don't have much experience with Go or the LLVM Go bindings so I
don't know any technical reasons for doing this but I do have a high
level question.
Why do this? In my experience having code hosted on GitHub (as llgo is
now) is quite nice because receiving contributions is much easier
(which hopefully means you will get more) and code review is simpler.
GitHub has lots of nice services (e.g. github.io pages, TravisCI and
Gitter) as well.
GitHub also has much better authentication (e.g. SSH keys and 2FA) and
repository separation where as LLVM is just one huge SVN repo where
everyone has access to every project and security isn't so good (e.g.
no 2FA, passwords are truncated to 8 characters because of the use of
CRYPT for generating the password hashes and SSLv3 is still in use).
The only advantages I can think of being a LLVM sub project are
* Your code is less likely to be silently broken by the fast passed
development of LLVM as it will trigger buildbot failures (assuming you
set one up for llgo).
* Being a sub project of LLVM gives the project more of an "official" status.
Are these the reasons enough to justify the move or are there reasons
that I've missed?
I think providing a better alternative to libffi is already a goal of
some people, but what is libbacktrace needed for? What is not provided
by the Itanium Unwind ABI?
A quick browse through the contributors on https://github.com/go-llvm/llgo/graphs/contributors and some searches in my
mail suggests that none of the primary contributors are part of the LLVM
community. How are they going to continue developing llgo once it is the
LLVM repo?
I am the original author of llgo, and FWIW I'm happy to work with the
LLVM infrastructure. Learning a new submission procedure isn't what's
going to hold me back, it's just lack of time right now.
Apart from Peter, there was really only one other who submitted a
significant amount of work, and he has not been involved for a fair
while now.
A quick browse through the contributors on https://github.com/go-llvm/llgo/graphs/contributors and some searches in my
mail suggests that none of the primary contributors are part of the LLVM
community. How are they going to continue developing llgo once it is the
LLVM repo?
The two main contributors are Andrew Wilkins and myself; I know that Andrew
is supportive of this move and presumably would continue to contribute in
the project's new location.
I don't know why most of my contributions are not showing on that page,
but I have contributed over 200 commits to llgo.
Sorry if this is a dumb question, but what is the advantage of having this
in-tree as an LLVM project? I typically associate the following advantages
to having something in-tree:
1. You can get the awesome code review of the LLVM community.
2. If people are changing the API, they have to fix in-tree stuff, so you
are shielded from ongoing maintenance burden.
Since the code is in go, you don't really get 1; the community speaks C++.
There is a subset of the community that is bilingual and willing to do code
review. As I mentioned, I am willing to be the code owner for this subproject,
so things will get reviewed.
IIRC the bindings are based on the C API, so they are stable and so 2
doesn't apply.
The bindings do use certain unstable APIs, such as debug info, and we may
decide to expand them to expose other unstable APIs. See also some of the
discussion in http://reviews.llvm.org/D5684
There are other advantages to having the project be part of LLVM. For example,
the standard library is built using llgo and a compatible version of Clang,
and this will be more important if we LTO the libgo library. At the moment,
llgo is built using a hackish set of shell scripts that make the correct
pieces of LLVM and Clang available; things are much simpler if everything
is built together.
The same goes for the sanitizer libraries in compiler-rt, which are fiddly
to use with llgo at the moment.
> llgo depends on certain third-party components, namely a copy of the Go
> standard library (libgo), a Go program analysis library (go.tools) and two
> library dependencies of the standard library (libbacktrace and libffi).
I think providing a better alternative to libffi is already a goal of
some people, but what is libbacktrace needed for? What is not provided
by the Itanium Unwind ABI?
According to the libbacktrace readme file:
As of September 2012, libbacktrace only supports ELF executables with
DWARF debugging information. The library is written to make it
straightforward to add support for other object file and debugging
formats.
Ian (the author of libbacktrace) may be able to provide more context.
I'm not sure I know the context for which I am providing context.
That said, libbacktrace is not an instantiation of the unwind ABI, it
is a client. The unwind ABI gives you a call stack but only provides
PC and SP values. libbacktrace gives you function names, file names,
and line numbers. It does this by reading the debug info at runtime;
at present it only supports DWARF.
I'd like to propose the contribution of a Go frontend subproject to the LLVM
project, based on the existing llgo project at https://github.com/go-llvm/llgo .
As with the previous contribution of the Go bindings, I have obtained
permission from all llgo contributors whose code is part of this contribution,
to contribute their changes to the LLVM project and relicense their changes
under the LLVM license. I am also willing to be the code owner for the
llgo subproject.
The frontend would live in the LLVM svn repository and if checked out into
tools/llgo would build as part of the regular LLVM build (with CMake only). We
would be keeping llgo compatible with top-of-tree LLVM, although I imagine
this would be less burdensome than the other subprojects as llgo is written
in Go and depends on the Go bindings previously contributed to LLVM.
llgo depends on certain third-party components, namely a copy of the Go
standard library (libgo), a Go program analysis library (go.tools) and two
library dependencies of the standard library (libbacktrace and libffi). These
would be mirrored into the llgo repository under a third_party directory. They
would retain their original licenses, which are BSD and GPLv3 with Runtime
Library Exception (the latter only applies to a handful of header files;
eventually we would seek to replace these).
Why do libbacktrace and libffi need to be mirrored in the llgo
repository? Can't llgo link with system versions of these libraries?
There are not necessarily going to be suitable system versions of these
libraries available. GCC seems to treat them more like implementation details
of their own runtime libraries. The libbacktrace library, for example, is
not installed on my machine in a central location, rather it appears in a
version-specific directory under /usr/lib/gcc.
The libffi that we use may also have Go-specific patches (e.g. [1]), so we
cannot use the system one.
Yes, but not only for that. libgo also uses the libbacktrace library
to implement the Go standard library function runtime.Caller, and
associated functions like runtime.Callers, runtime.FuncForPC,
runtime.Stack, etc.
I must profess a dislike for the trend of corporate languages-as-platforms that appears to have spawned the Go project, along with several other recent languages. Based on the disaster Oracle has made of Java, I think that having a single corporation behind a language and not working toward standardization is not a healthy sign.
That you are working on a properly-licensed open source frontend is a great start in a positive direction. I want to understand more about Go in order to understand the issues that will come from integrating this frontend. Put another way -- this is an opportunity for you to evangelize the language, teaching us old C fuddy-duddies all about Go and why it deserves our attention. What is it being used for? How widespread is the adoption? Is there some TL;DR reading out there we should focus our time on? Is there an equivalent of Gary Berhardt's excellent WAT presentation that addresses the gotchas in the language design that we should also be aware of?
A good recent summary of the language and its community is found here:
You will see from that article that a number of companies and open source
projects have adopted the language. If you follow the links there you can
also read more about the philosophy behind the language.
I also just wanted to clarify that the 'gc' implementation of the Go language,
which is the one maintained by the Go team at Google and the one that most
people use, is BSD licensed, and most of the runtime library code that
llgo uses is based on that implementation. There are other reasons besides
licensing why one would want to use the llgo frontend
Just curious: are you considering the possibility of enabling LTO across user and runtime code?
Since AFAIK the product of go build is almost always a statically-linked executable, I guess it would make sense (when doing an optimised build) to feed the runtime to the linker in IR form and let it run LTO.
Just curious: are you considering the possibility of enabling LTO across user and runtime code?
Since AFAIK the product of go build is almost always a statically-linked executable, I guess it would make sense (when doing an optimised build) to feed the runtime to the linker in IR form and let it run LTO.
