std::cout << *MyModule does not work anymore

It seems that support for dumping text representation of LLVM objects to
standard channels and C++ output streams was removed. My guess is that
now we must use errs() instead of std::cerr, llvm::raw_fd_ostream
instead of std::ofstream, etc.

The changes are not trivial, as for instance llvm::raw_fd_ostream
without flags fails if the file exists, but std::ofstream does not. The
changes include using new names for flags that already exist on the
standard namespace (F_Force instead of O_TRUNC, etc).

Is all this an unintended change or an intentional one, and if the
later, could you direct me to something that explains what we gain on
exchange of this restriction?

It seems that support for dumping text representation of LLVM objects to
standard channels and C++ output streams was removed. My guess is that
now we must use errs() instead of std::cerr, llvm::raw_fd_ostream
instead of std::ofstream, etc.

Correct. std::ostream has been purged from LLVM; the only exception is
the raw_os_ostream class, which can be used to adapt an std::ostream
for use with raw_ostream. This should be enough for clients which want
to keep using std::ostream.

The changes are not trivial, as for instance llvm::raw_fd_ostream
without flags fails if the file exists, but std::ofstream does not. The
changes include using new names for flags that already exist on the
standard namespace (F_Force instead of O_TRUNC, etc).

I believe that Dan plans to fix the "force by default" issue, at least.

Is all this an unintended change or an intentional one, and if the
later, could you direct me to something that explains what we gain on
exchange of this restriction?

Intentional. std::ostream has long been deprecated in LLVM, each
release has increased the degree of deprecation. The coding standards
document has various bits of information on this
http://llvm.org/docs/CodingStandards.html.

Among other things, you gain performance. raw_ostream is considerably
faster than std::ostream. This is the performance difference on the my
synthetic raw_ostream benchmark:
- raw_fd_ostream (outs()) -

Daniel Dunbar <daniel@zuster.org> writes:

[snip]

Is all this an unintended change or an intentional one, and if the
later, could you direct me to something that explains what we gain on
exchange of this restriction?

Intentional. std::ostream has long been deprecated in LLVM, each
release has increased the degree of deprecation. The coding standards
document has various bits of information on this
http://llvm.org/docs/CodingStandards.html.

Among other things, you gain performance. raw_ostream is considerably
faster than std::ostream. This is the performance difference on the my
synthetic raw_ostream benchmark:

[snip]

First of all, thanks to Daniel for the answer to my questions.

To the LLVM library designers:

Okay, so your wheel is lighter and faster. Sometimes this is a good
justification for reinventing the wheel when everybody is eager to throw
away the old wheels and use yours instead :-). However, when LLVM is
just another library on an application (something that I hope often
happens thanks to the great JIT it provides) having to learn the quirks
and kinks of something that pretends to replace a common functionality
of the language's standard library *plus* integrating it into the
existing code can be a real burden. Maybe the LLVM ostream replacement
knows how to handle LLVM objects, but it certainly knows nothing about
how to deal with user's objects and this may create quite a bit of work
when you have a mixed series of output statements with both kinds of
objects.

Maybe there was a solution that was okay for those concerned with fast &
lightweight executables and for the others who care most about
consistency and code stability. I would appreciate if this changes are
discussed before applying them on the future (maybe I missed the
thread), flagging the subject with something that indicates that the
outcome of the discussion may break existing code.

Thanks for your mostly excellent work on LLVM's design and policy-making
:wink:

However, when LLVM is
just another library on an application (something that I hope often
happens thanks to the great JIT it provides) having to learn the quirks
and kinks of something that pretends to replace a common functionality
of the language's standard library *plus* integrating it into the
existing code can be a real burden.

Right, this is exactly why we have raw_os_ostream. If you have an std::ostream, you can can make LLVM output to it by using raw_os_ostream. This way you get the internal performance wins in llvm plus the compatibility with std::ostream. Bonus, it should faster than writing to std::ostream directly.

Maybe there was a solution that was okay for those concerned with fast &
lightweight executables and for the others who care most about
consistency and code stability. I would appreciate if this changes are
discussed before applying them on the future (maybe I missed the
thread), flagging the subject with something that indicates that the
outcome of the discussion may break existing code.

raw_ostream has been a long time coming, it was started in 2008-08-16 18:35:29. The coding standards doc has *long* had wording that <iostream> (in particular) is evil.

-Chris

Hi Óscar,

Maybe there was a solution that was okay for those concerned with fast &
lightweight executables and for the others who care most about
consistency and code stability. I would appreciate if this changes are
discussed before applying them on the future (maybe I missed the
thread), flagging the subject with something that indicates that the
outcome of the discussion may break existing code.

hopefully the 2.6 release notes will mention these changes and how
to handle them (I don't think anyone added anything yet though).

Ciao,

Duncan.

Óscar Fuentes wrote:

The changes are not trivial, as for instance llvm::raw_fd_ostream
without flags fails if the file exists, but std::ofstream does not. The
changes include using new names for flags that already exist on the
standard namespace (F_Force instead of O_TRUNC, etc).

Also, each of LLVM <=2.5, 2.6 and 2.7(svn) provide their own,
incompatible llvm::raw_fd_ostream constructors. This makes it
unneccessarily hard to support different LLVM versions in a frontend.

I understand why the llvm::raw_fd_ostream interface was changed, but it
would have been nice if the old constructors were kept and implemented
in terms of the new ones. That's completely trivial in this case and
shouldn't cause any harm in existing applications. For many applications
other than LLVM itself, the basic LLVM <=2.5 interface which just
overwrites existing files is all that's ever needed.

Albert

On LLVM trunk, raw_fd_ostream is now back to overwriting files by
default, as it is the unanimous preference among in-tree users (and
out-of-tree users that I'm aware of).

Dan

FWIW, the changes under discussion here are on trunk and not 2.6.

Dan

Dan Gohman wrote:

. For many applications
other than LLVM itself, the basic LLVM <=2.5 interface which just
overwrites existing files is all that's ever needed.

On LLVM trunk, raw_fd_ostream is now back to overwriting files by
default, as it is the unanimous preference among in-tree users (and
out-of-tree users that I'm aware of).

But the raw_fd_ostream constructor is *still* incompatible with both
LLVM <=2.5 and LLVM 2.6:

LLVM 2.5:
  raw_fd_ostream(const char *Filename, bool Binary,
                 std::string &ErrorInfo);

LLVM 2.6:
  raw_fd_ostream(const char *Filename, bool Binary, bool Force,
                 std::string &ErrorInfo);

Trunk (r80020):
  raw_fd_ostream(const char *Filename, std::string &ErrorInfo,
                 unsigned Flags = 0);

It would be helpful to emulate the LLVM 2.5 variant of the constructor
on both 2.6 and trunk, so that frontend developers don't have to code
against three different versions of the constructor, if they don't need
the added functionality.

Albert

We do not guarantee API stability at all, so this is just the tip of the iceberg. At one point we discussed having a Version.h that you could #include and then #ifdef based on the version number, I don't know what happened to it if anything.

It seems that it would be relatively easy to get autoconf to make a include/llvm/Config/Version.h file that did this.

-Chris

Chris Lattner wrote:

We do not guarantee API stability at all, so this is just the tip of the
iceberg.

I understand that. But in this specific case it would be very easy to
maintain, no?

It seems that it would be relatively easy to get autoconf to make a
include/llvm/Config/Version.h file that did this.

I already have the necessary autoconf magic to deal with this in my
frontend. I just hoped that I might get rid of it in this case. All
those #ifdef's don't make the code prettier. :slight_smile:

Albert

Albert Graef wrote:

I understand that. But in this specific case it would be very easy to
maintain, no?

FWIW, suggested patch attached. That makes the LLVM 2.5 style
raw_fd_ostream constructor work with the trunk (r80036). Note that this
just adds a second constructor, existing code will continue to work as
before. I'd be happy to prepare an analogous patch for the 2.6 release
branch, just let me know.

Albert

raw_fd_ostream.patch (2.8 KB)

Hi Albert,

Given that we don't guarantee backwards compatibility and prefer to keep our APIs clean and tidy, why should we take this?

-Chris

Chris Lattner wrote:

Given that we don't guarantee backwards compatibility and prefer to keep
our APIs clean and tidy, why should we take this?

I already said that: To make it easier for out-of-tree frontends to
support at least the last few LLVM versions. If that's a low priority
then don't. I can live with it.

Please understand that I don't complain about API breakage in general.
It was certainly necessary to implement the multithreading support and
some other stuff in 2.6, and in any case it's a good thing that LLVM
develops new functionality so fast.

In this specific case, however, I think that the API breakage was
completely unneccessary and could have easily been avoided right from
the beginning. In fact, you could have just taken the LLVM 2.5
raw_fd_ostream constructor, turned the 'bool Binary' into an 'unsigned
flags' (with F_Binary=1 for backward compatibility, other flags added as
needed) and old code would have just continued to work and new code
would have been happy, too. But that's not what happened. Instead we now
have a mess of three different raw_fd_ostream interfaces with almost
identical functionality in three consecutive LLVM versions. My patch
just tries to correct this as good as possible given that the API was
already changed in an incompatible way.

Well, that's my take on it. At least I tried. :slight_smile:

Albert

Chris Lattner wrote:

Given that we don't guarantee backwards compatibility and prefer to keep
our APIs clean and tidy, why should we take this?

I already said that: To make it easier for out-of-tree frontends to
support at least the last few LLVM versions. If that's a low priority
then don't. I can live with it.

I don't understand how that can be. We have made *many dramatic API changes* in those releases, surely this isn't the only problem you have hit. Are you using the C++ APIs? We try to keep the C apis as stable as possible, is it possible to use those?

Just to be clear, I'm not trying to be a butthead about this :). I just see that API stability in almost a binary way: you either try to keep it or not. If not, I don't think that trying to paper over small changes like this is worth it.

In fact, you could have just taken the LLVM 2.5
raw_fd_ostream constructor, turned the 'bool Binary' into an 'unsigned
flags' (with F_Binary=1 for backward compatibility

Sure, but again, the change wasn't made for an arbitrary reason: moving them to the end allows them to be optional, so that they do not need to be specified. This makes the API more elegant. I'm not willing to sacrifice small bits of API elegance unless we actually get something useful out of this. Given the other massive and widespread api changes in 2.6, I can't fathom how this can actually matter.

-Chris

Mmm... I agree in general, but I think it's worth considering that
this change could silently hit someone updating from an older version
of LLVM.

-Eli

Oh, wait, nevermind... I wasn't reading this thread carefully enough.

-Eli

Chris Lattner wrote:

I don't understand how that can be. We have made *many dramatic API
changes* in those releases, surely this isn't the only problem you have
hit.

Surely not. :slight_smile:

Are you using the C++ APIs?

Yes.

We try to keep the C apis as stable as possible, is it possible to use those?

No, but that's good to know. I might want to generate a Pure-LLVM
binding via the C API some time.

Just to be clear, I'm not trying to be a butthead about this :). I just
see that API stability in almost a binary way: you either try to keep it
or not. If not, I don't think that trying to paper over small changes
like this is worth it.

You said it. :slight_smile: Case closed. I've solved this already in my configury
anyway.

Sure, but again, the change wasn't made for an arbitrary reason: moving
them to the end allows them to be optional, so that they do not need to
be specified. This makes the API more elegant.

I understand that.

Albert