a few of us had recently a discussion about how to manage the C API and possible policies regarding addition, maintenance, deprecation, and removal of API.
Even thought there is a strong agreement in the community that we shouldn't break released C API and should be backwards compatible, there doesn’t seem to be a developer policy that backs that up. This is something we should fix.
I was wondering what the interested parties think of the current approach and what could/should we improve to make the use and maintenance of the C API easier for users and the developers alike.
I was hoping we could also introduce a process that allows the removal of an API after it has been deprecated for a whole release and the release notes stated that it will be removed.
I definitely agree with needing to write down the policy and definitely any policy should include a deprecation time
Did you want to write up something and let us poke at it?
A couple of thoughts outside of a deprecation policy:
a) guarantee that the api itself won’t go away, but could possibly be turned into a noop? (i.e. source compatibility)
b) versioning? (Should we add support for versioning in anyhow? Maybe a C API version 2 to start this off with?)
c) One of the big things appears to be the push and pull for “A C API for all C++ entry points” along with “We don’t want to get locked into immobility because we have a C API for all C++ entry points”. Perhaps part of this might be defining an actual stable set of things along with an unstable set of things?
What was the rest of your discussion on this? Any particular points you think the community would be interested in?
I’m totally ok with a small first patch that puts in a developer policy of “we try to keep the C API stable across releases and the deprecation policy is blah blah blah”
c) One of the big things appears to be the push and pull for "A C API for
all C++ entry points" along with "We don't want to get locked into
immobility because we have a C API for all C++ entry points". Perhaps part
of this might be defining an actual stable set of things along with an
unstable set of things?
This is a good idea. We should clearly document a subset that is stable.
Being stable should also have a very high bar. Things like "we have a
shipping product that has to work with two versions of llvm and has to
dynamic link with it".
So something that the webkit jit needs is in. Some API that is in C
just so someone can statically link a C/Go/Ocaml program with llvm is
not.
c) One of the big things appears to be the push and pull for “A C API for
all C++ entry points” along with “We don’t want to get locked into
immobility because we have a C API for all C++ entry points”. Perhaps part
of this might be defining an actual stable set of things along with an
unstable set of things?
This is a good idea. We should clearly document a subset that is stable.
Being stable should also have a very high bar. Things like “we have a
shipping product that has to work with two versions of llvm and has to
dynamic link with it”.
So something that the webkit jit needs is in. Some API that is in C
just so someone can statically link a C/Go/Ocaml program with llvm is
not.
This is pretty open to abuse, let’s just define what seems to make sense rather than tying it like this.
a few of us had recently a discussion about how to manage the C API and possible policies regarding addition, maintenance, deprecation, and removal of API.
Even thought there is a strong agreement in the community that we shouldn't break released C API and should be backwards compatible, there doesn’t seem to be a developer policy that backs that up. This is something we should fix.
+1
I was wondering what the interested parties think of the current approach and what could/should we improve to make the use and maintenance of the C API easier for users and the developers alike.
I was hoping we could also introduce a process that allows the removal of an API after it has been deprecated for a whole release and the release notes stated that it will be removed.
+1
I'd suggest we also have an officially unofficial policy about not versioning just for style or cleanliness reasons. i.e. We should try to minimize churn of the API unless it's actually needed, or supporting an old API becomes unjustifiably complicated.
c) One of the big things appears to be the push and pull for "A C API for
all C++ entry points" along with "We don't want to get locked into
immobility because we have a C API for all C++ entry points". Perhaps part
of this might be defining an actual stable set of things along with an
unstable set of things?
This is a good idea. We should clearly document a subset that is stable.
Being stable should also have a very high bar. Things like "we have a
shipping product that has to work with two versions of llvm and has to
dynamic link with it".
+1 to the general notion of stable vs unstable
So something that the webkit jit needs is in. Some API that is in C
just so someone can statically link a C/Go/Ocaml program with llvm is
not.
Can we also codify when something should be added to the C API? For a
lot of folks the C API is the only usable interface. I am one of them.
We are not as vocally represented because don't generally give back to
the community, usually because we are just consumers of this library.
(Or maybe I'm totally wrong and lots of us give back).
Could you explain what you mean here? As far as I can tell this is “I don’t want versioning unless I want versioning and then I’ll want it because it’s convenient for me”
I’ve actually got another, perhaps more radical, plan. Let’s just get rid of the C API or move it to another project. This simplifies a lot of the plans here where people have too many different ideas of how the C API should work.
At this point the people who want a stable C API per incremental version can do that and handle the overhead of porting themselves and the people that want a C API that just happens to be a C interface can have a wrapper (or SWIG or whatever they want).
I realize it’s radical, but it seems that there are so many different wants for C API here that solving everyone’s problems or wants is going to be impossible.
What I was trying to get at is that we should aim to not change the API unless it’s actually needed. Just because we have a versioning mechanism doesn’t mean we should freely make use of it. Some examples: - Renaming a method in the API - bad - Dropping a method involving functionality no longer supported - good We tend to be much more free with the C++ APIs (with good cause). I don’t want to see this applied to the C API. We should still seek to keep a stable API even if we do have a formal mechanism for revising it. p.s. If that sounds obvious, you’ve read it right. This wasn’t intended to be controversial, just to cement in writing the attitude we’ve already taken. p.p.s. The preceding would only apply to the “stable” parts of the API. Philip
I really haven’t seen that much of a split here honestly. Everyone agrees we need a stable C API for core functionality. The only disagreement seems to be about when something gets promoted to “core” status and even that’s been minimal. (I have no opinion w.r.t. your actual proposal.)
I've actually got another, perhaps more radical, plan. Let's just get rid
of the C API or move it to another project. This simplifies a lot of the
plans here where people have too many different ideas of how the C API
should work.
At this point the people who want a stable C API per incremental version
can do that and handle the overhead of porting themselves and the people
that want a C API that just happens to be a C interface can have a wrapper
(or SWIG or whatever they want).
I realize it's radical, but it seems that there are so many different
wants for C API here that solving everyone's problems or wants is going to
be impossible.
I am strongly in favor of moving the bindings, C or otherwise, to another
project.
This is interesting. If we did this, the same would apply to the other language bindings, too, yes? We’d end up with a few different mini-projects (or possibly just one w/ a configurable build system).
It would also set the stage for, potentially, multiple sets of bindings. As Eric points out, one size doesn’t always fit all, and our current path of trying to find some sort of happy medium may not be the best. This would allow us to have one, or more, binding set which map to different use cases appropriately.
So long as we set things up in a way that makes it easy to configure both a unified and discrete (in tree vs. using a “make install” artifact) build, had good bot coverage, etc, etc, I think this could work really well.
That said, it’s also a bit of a derail from and most orthogonal to Juergen’s main topic. Perhaps we should split this off to a different thread?
I've actually got another, perhaps more radical, plan. Let's just get rid
of the C API or move it to another project. This simplifies a lot of the
plans here where people have too many different ideas of how the C API
should work.
At this point the people who want a stable C API per incremental version
can do that and handle the overhead of porting themselves and the people
that want a C API that just happens to be a C interface can have a wrapper
(or SWIG or whatever they want).
I realize it's radical, but it seems that there are so many different
wants for C API here that solving everyone's problems or wants is going to
be impossible.
This might backfire if this results in a much better tested interface
between the C++ and C API implementation, such that we will actually get
red bots when we "break the C API" and be forced to fix it.
+1 for Eric’s idea, as it would solve my problem with providing a C API for Orc: Being a templated and component-based library, there’s no definitive Orc JIT configuration to provide a C API for. Separate projects would allow different clients to pick an API that makes sense for them.
I definitely agree with needing to write down the policy and definitely any policy should include a deprecation time
Did you want to write up something and let us poke at it?
A couple of thoughts outside of a deprecation policy:
a) guarantee that the api itself won’t go away, but could possibly be turned into a noop? (i.e. source compatibility)
This was done recently with the Go bindings to make the LLVM tree green, which led to not-immediately-obvious breakage in llgo. If source compatibility were broken, it would have been clear what needed to be changed. As a user, I’d prefer that over preserving source compatibility.
b) versioning? (Should we add support for versioning in anyhow? Maybe a C API version 2 to start this off with?)
c) One of the big things appears to be the push and pull for “A C API for all C++ entry points” along with “We don’t want to get locked into immobility because we have a C API for all C++ entry points”. Perhaps part of this might be defining an actual stable set of things along with an unstable set of things?
IMO, just having a single stable C API for the core would keep things clear; no having to check whether this function or that function is stable. Separate non-stable APIs can be maintained outside the core easily enough. They don’t even need to be in-tree if they’re not stable.
