HTTP library in LLVM

We’re considering implementing debuginfod library in LLVM. Initially, we’d like to start with the client implementation, which would enable debuginfod support in tools like llvm-symbolizer, but later we’d also like to provide LLVM-based debuginfod server implementation.

debuginfod uses HTTP and so we need an HTTP library, ideally one that supports both client and server.

The question is, would it be acceptable to use an existing C++ HTTP library or would it be preferred to implement an HTTP library in LLVM from scratch?

[+debug info folks, just as FYI - since the immediate question’s more about 3rd party library deps than the nuances of DWARF, etc]

I’d imagine avoiding writing such a thing from scratch would be desirable, but that the decision might depend somewhat on what libraries out there you/we would consider including, what their licenses and further dependencies are.

+LLDB Dev as well for visibility. +Pavel Labath since he and I have talked about such things.

There are several options, I’ve looked at couple of them and the one I like the most so far is https://github.com/yhirose/cpp-httplib for a few reasons:

  • It’s MIT licensed.

  • It supports Linux, macOS and Windows (and presumably other platforms).

  • It doesn’t have any dependencies, it can optionally use zlib and OpenSSL.

  • It’s a modern C++11 implementation, the entire library is a single header.

There are several options, I’ve looked at couple of them and the one I like the most so far is https://github.com/yhirose/cpp-httplib for a few reasons:

  • It’s MIT licensed.

I hesitate to get into it on the list, not-a-lawyer, etc. But does that seem like it’d be as usable as other code we have (zlib, gtest, etc) used by/in LLVM?

  • It supports Linux, macOS and Windows (and presumably other platforms).

  • It doesn’t have any dependencies, it can optionally use zlib and OpenSSL.

  • It’s a modern C++11 implementation, the entire library is a single header.

Handy - I guess you’d want to check that in (ala gtest, rather than ala zlib which is used from the system) to the llvm-project repository, then?

There are several options, I’ve looked at couple of them and the one I like the most so far is https://github.com/yhirose/cpp-httplib for a few reasons:

  • It’s MIT licensed.

  • It supports Linux, macOS and Windows (and presumably other platforms).

  • It doesn’t have any dependencies, it can optionally use zlib and OpenSSL.

  • It’s a modern C++11 implementation, the entire library is a single header.

This looks appealing indeed. Out of curiosity, what are the other alternatives you considered?

These are the ones I found so far:

  • libmicrohttpd is used by elfutils’ debuginfod, but it’s LGPL licensed.

  • libcURL would be an option for the client, but we’d need a different library for the server.

  • libhttp is another MIT licensed library that could be a fit, but it seems bigger and more featureful than httplib.

  • cpprestsdk has a lot of extra features we don’t need, like websockets.

  • pistache similarly has additional features and dependencies that are likely unnecessary.

  • crow is similar to cpprestsdk and pistache, it depends on Boost.

  • cpp-netlib looks nice but depends on Boost.

  • proxygen is also nice but has a lot of dependencies including Boost.

Hello,

I have only negative experiences with cpp-netlib - I would not recommend it.

If the discussion was around only the client part I think libcURL
would be the best option since it's available everywhere and is a well
maintained library. But as you mention it's only the client part.
We use pion (https://github.com/splunk/pion) to build a HTTP server
interface, but since it seems pretty unmaintained I am not sure I
would recommend it.

I have heard good things about https://github.com/matt-42/lithium/ but
have no direct experience of using / building it.

Thanks,
Tobias

Hello,

I have only negative experiences with cpp-netlib - I would not recommend it.

Can you elaborate on those negative experiences? I think that would be helpful in comparing it to the other alternatives.

If the discussion was around only the client part I think libcURL

would be the best option since it’s available everywhere and is a well

maintained library. But as you mention it’s only the client part.

Yeah, I thought the same going through the list.

I have only negative experiences with cpp-netlib - I would not recommend it.

Can you elaborate on those negative experiences? I think that would be helpful in comparing it to the other alternatives.

Sure:
* It has not had a release since 2013 and the current release is not
compatible with current boost. You can go and use the master branch
but it's not that much more maintained.
* It depends on boost and specifically boost.spirit to do parsing
which is almost impossible to debug and is a VERY heavy dependency to
compile.
* it's not part of upstream boost but it installs it's headers into
the boost directory by default and pretends to be a boost library. (I
guess this is because it was at some point slated to be part of
boost).

TBH - I would be very hesitant to add a library that depends on boost,
while it has a lot of nice features and code it's a very heavy
dependency for LLVM to add at this point.

Hope this helps.
Tobias