Preferring to use GCC instead of LLVM

Project Infrastructure LLVM Dev List Archives
1

Jon Harrop wrote:

Can you explain why you would like to generate DLLs on the customer's computer rather than using LLVM as a JIT compiler?

Customers/clients unhappy with the inefficiency, extra CPU and RAM usage, and performance penalty of JIT. They require a faster, more efficient solution. The solution is to fully compile programs to native code at the time of installation of programs on customer's computer, instead of incurring JIT overhead every time the program is launched and run by customer.

A customer installs a program once (or rarely), but launches/runs it many times. Customers don't care if program installation (including compiling to native code) is a bit slow, because they do it only once. But they DO care if launching and/or running the program is slow or inefficient, because they do this very often.

Customers/clients demand that cost of converting to native code be shifted out of run-time, and into installation-time instead.

Major client says using JIT is "a moronic strategy with unnecessary complexity and undesirable performance (CPU and RAM use) characteristics, with significant disadvantages compared to fully converting programs to native code at time of installation".

Nick Lewycky wrote:

Although we can emit the opcodes in binary form, support for the container formats isn't working properly. Thus far, there's an unfinished ELFWriter and MachOWriter in lib/Codegen/. Those could be finished, and PEWriter could be added for Win32 support.

It would be wonderful if support for the 3 container formats could be finished/implemented. It would make LLVM far more appealing for use in real-world situations.

If this is blocking you from using LLVM,

Yes, it is.

please consider whether it'd be easier to write this for us than to use something else.

Would love to, but unable to do that this year. Might be able to do it this time next year at the earliest. Also would love to throw money at certain LLVM developers, but not in a position to do that at the present time, maybe later.

Bill Wendling wrote:

GCC generates assembly code too and calls the assembler and linker as part of it's execution. You are perfectly able to call the assembler & linker yourself.

No, the assembler and linker do not exist on customer's computers running Windoze.

Bill Wendling wrote:

You should really learn how to use Google. Got this as the top hit for "Microsoft Assembler": [Microsoft Macro Assembler 8.0 (MASM) Package (x86)]

Incorrect. That webpage says that it requires that Microsoft Visual C++ 2005 (Express Edition) be installed on the computer before MASM can be installed.

We cannot reasonably require our customers to install Microsoft Visual C++, or if we did, then we may as well use MS Visual C++ to do everything -- then there is no need for LLVM.

Oscar Fuentes wrote:

This means that LLVM requires an assembler and linker. Call it GCC or binutils, it is irrelevant. The OP point is that LLVM is not a self-sufficient tool on this aspect.

Correct.

Owen Anderson wrote:

The OP was implying that LLVM is incomplete because it depends on GCC in the backend, which is incorrect. It depends on binutils,

Alright, so I used the wrong name. I said GCC but I should have said binutils. My original point remains valid -- LLVM is an incomplete backend because by itself it is unable to generate a ready-to-execute .EXE or .DLL file.

And binutils is not necessarily available on the customer's computer. Do all Linux and BSD distributions include it preinstalled? Maybe, but what about Windoze and MacOS? I know Windoze does not include it or any equivalent. And Apple's developer tools (Xcode etc) for MacOS X are not preinstalled either, they are a separate download (although perhaps binutils is preinstalled in MacOS X, not sure, haven't checked).

Now, if I am going to require that the customer download and install Apple's Xcode package, or Microsoft's C compiler tools, then I may as well use them to do everything I need. Having them installed eliminates the need for LLVM. But I would prefer to use LLVM, and I certainly don't want to require that customers download and install Apple's or Microsoft's compiler tools.

Owen Anderson wrote:

I think the point is that implementing our own linker is definitely beyond the purview of LLVM, and implementing our own object file writer/assembler is honestly pretty low on the priority queue.

That attitude is very much hurting LLVM's popularity and adoption in real-world situations.

Owen Anderson wrote:

From what I read, [MASM is] included with copies of Visual Studio. I assume that a paid copy of Visual Studio (as opposed to the free Express edition) includes a version that's licensed for commercial use.

Then we may as well use Visual Studio to do everything, and not bother with LLVM. But I prefer to use LLVM if possible.

Additionally, LLVM's support for Win32 outside of MinGW/Cygwin is ... spotty at best.

LLVM needs to sever its backend ties with MinGW/Cygwin/GCC/binutils. Because if we are going to use the hacked bastard child of LLVM combined with MinGW/Cygwin/GCC/binutils, then we may as well just use the normal version of MinGW/Cygwin with GCC and not bother with the added complication of LLVM.

The fundamental problem is that both executable formats and linking problems are EXTREMELY platform specific, so it's typically best to leave it up to the system tools.

But these system tools do not exist (or are not installed) on all relevant/significant customer computers.

P.S. I do agree that, if he has a desperate need to run LLVM outside of MinGW/Cygwin, JIT is his best bet.

JIT is a bad solution, for the reasons I explained above.

Oscar Fuentes wrote:

For a Windows folk, having extra requirements goes against what he is accustomed to. If LLVM is perceived as a backend, but it fails short into producing the final result, the typical Windows folk will be disappointed.

Yes. Very much so.

The OP talks about GCC because that's what he sees, irrespectively of the other tools behind.

Yes, I used the wrong name, but my point is still valid.

Oscar Fuentes wrote:

If the OP plans to deploy his compiler on Windows, he can't assume the existence of an assembler/linker on most user's machines. He becomes dependent of third-party packages, hence LLVM is not a "complete backend" from his (limited) point of view.

Yes!! I know you people like Linux and BSD, and I do too, but the fact is we have customers using Windoze, and we have customers using Linux, and we need to support them both.

To say that we should ditch all our Windoze-using customers and only support Linux-using customers, to say that would be prejudiced, fanatical, irrational, unrealistic, unreasonable, and living in a fantasy dream world detached from reality.

Expect more messages like this as LLVM becomes more popular outside professional and academical circles.

Yes. But also as LLVM becomes more popular inside professional, commercial, and business circles focused on delivering real-world practical implementations/solutions.

Chris Lattner wrote:

FWIW, I wouldn't be surprised if the LLVM project eventually grew more of native toolchain support (e.g. assembler, linker, etc). It would fit naturally with the growing scope of the project.

Yes, it would, and it would make LLVM more popular and more usable in real-world situations.

I don't think that it is a high priority for anyone though.

Then you are disadvantaging the LLVM project and holding it back. I see you are working on a C front-end "clang". Personally I think that a C front-end should be a much lower priority than delivering a COMPLETE backend.

Another C compiler is not needed. GCC and MSVC already do that job, and you won't supplant them. The thing that is really needed is in accordance with the essence of LLVM -- a complete backend solution.

Emilio Wuerges wrote:

Is this thread suposed to be a bad joke?

That is a very unprofessional response to my legitimate questions and concerns, and my good-faith attempts to ASSIST the LLVM project by providing some constructive criticism and feedback (and possibly contributions later).

Terence Parr wrote:

Yeah, i'm kinda freaked out as i'm sure it's not a joke. i'm in awe. +1 to kill thread before google spiders it.

Again, that is a very unprofessional response.

Bill Wendling:

OTOH, I'm curious about why the OP needs to produce dlls on the fly. Isn't the JIT ok?

I doubt he knows.

Doubt I know? Another unprofessional response. I have now answered Jon's JIT question in this email.

Chris Lattner wrote:

If you'd prefer to use GCC, go for it. No one is forcing you to use LLVM.
You are seriously ignorant of what LLVM is all about. Please go inform yourself.

That was an unprofessional response as well.

2

No, it doesn't. You said that if I have to install GCC, you might as
well just use it for everything. That statement very clearly doesn't
apply anymore, since it's binutils that's the dependency. Or if you
still stand by it, it means that you consider GCC to also be
"incomplete".

3

Owen Anderson wrote:

The OP was implying that LLVM is incomplete because it

depends on GCC in the backend, which is incorrect. It

depends on binutils,

Alright, so I used the wrong name. I said GCC but I should
have said binutils. My original point remains valid – LLVM
is an incomplete backend because by itself it is unable to
generate a ready-to-execute .EXE or .DLL file.

By your definition, GCC is incomplete as well. Of course, the solution that is typically employed for GCC installations is to bundle the necessary parts of binutils with it, for instance in Apple’s developer tools package. There’s nothing particularly stopping you from having your installation package include copies of gas and ld, and would result in an LLVM that is just as “complete” as GCC.

And binutils is not necessarily available on the customer’s
computer. Do all Linux and BSD distributions include it
preinstalled?

No, they don’t. I know, for example, that Ubuntu doesn’t. But in its packaging mechanism (apt), it pulls them in as a dependency when you install GCC. The appropriate thing to do would be to make them a dependency of LLVM as well.

Owen Anderson wrote:

I think the point is that implementing our own linker is

definitely beyond the purview of LLVM, and implementing

our own object file writer/assembler is honestly pretty

low on the priority queue.

That attitude is very much hurting LLVM’s popularity and
adoption in real-world situations.

You’re welcome to think that, and I’m welcome to think that your demands are unrealistic.

When I write a text editor, do I first need to write an OS kernel to run it on? How about disk device drivers? Display drivers? Of course not. I build on tools that already exist. Similarly, compilers build on other tools like linkers and assemblers. Of course, they typically hide this dependency by including the linker/assembler inside their installation package. Which is exactly what you should be doing.

The fundamental problem is that both executable formats

and linking problems are EXTREMELY platform specific, so

it’s typically best to leave it up to the system tools.

But these system tools do not exist (or are not installed)
on all relevant/significant customer computers.

Incorrect. They exist for all of the relevant platforms, or else development on that platform would be impossible/extremely difficult. You just need to distribute them as part of your installation package (or via a dependency mechanism on systems like Ubuntu that use them).

LLVM needs to sever its backend ties with
MinGW/Cygwin/GCC/binutils. Because if we are going to use
the hacked bastard child of LLVM combined with
MinGW/Cygwin/GCC/binutils, then we may as well just use the
normal version of MinGW/Cygwin with GCC and not bother with
the added complication of LLVM.

See, this is the part where you throw people. You keep stating that you really want to use LLVM, but this kind of statement makes it sound like you don’t really care.

I’m going to let you in on a reality of open source development: if you really particularly want some feature, you might have to end up writing it yourself. This is true whether you’re talking about LLVM, GCC, the Linux kernel, or emacs. Showing up and demanding that they be done for you isn’t going to get you anywhere, particularly since plenty of users seem to get along fine w/o what you’re demanding.

In all sincerity, I hope you get something that will work for you, whether it’s LLVM or something else. But I do think your expectation that we’re all going to drop what we’re doing in order to embark on multi-year projects to replace underlying system tools when plenty of other users have figured out how to work with the existing ones is unrealistic.

Good luck,

–Owen

4

Chris Lattner wrote:

FWIW, I wouldn't be surprised if the LLVM project
eventually grew more of native toolchain support (e.g.
assembler, linker, etc). It would fit naturally with the
growing scope of the project.

Yes, it would, and it would make LLVM more popular and more
usable in real-world situations.

Sure, there would be some people who would like that. However, you seem to be under the mistaken impression that contributors to LLVM share you sense of priority. It is not like there is any shortage of things to do here. Who is to say that providing a "complete backend" is more important than improving the code generator to produce faster code? For some people, faster code is far more important than providing a "complete backend". Who is to say that your priorities are more important than they are?

Ultimately it boils down to who contributes code, and/or who pays others to contribute code. If you would like to see LLVM have a "complete backend" (again, by your definition) you should contribute code to make it happen, or pay someone to do so. Telling people that it is *now* their highest priority (because *you* say so) is extremely discourteous, and unlikely to produce results in your favor.

I don't think that it is a high priority for anyone
though.

Then you are disadvantaging the LLVM project and holding it
back. I see you are working on a C front-end "clang".
Personally I think that a C front-end should be a much lower
priority than delivering a COMPLETE backend.

Another C compiler is not needed. GCC and MSVC already do
that job, and you won't supplant them. The thing that is
really needed is in accordance with the essence of LLVM -- a
complete backend solution.

This is a great example of you thinking that your set of priorities is higher than "my" set of priorities. How do you know what is important to me, and why do you think it is ok to have the gall to tell me that my priorities are wrong? You notice that I contribute code to make my priorities happen, unless you do the same, it is unlikely that your notion of a "complete backend" will ever magically materialize, despite you arguing with people.

Chris Lattner wrote:

If you'd prefer to use GCC, go for it. No one is forcing
you to use LLVM.
You are seriously ignorant of what LLVM is all about.
Please go inform yourself.

That was an unprofessional response as well.

How so? What aspect is untrue?

-Chris

5

Chris Lattner wrote:

FWIW, I wouldn't be surprised if the LLVM project eventually grew more of native toolchain support (e.g. assembler, linker, etc). It would fit naturally with the growing scope of the project.

Yes, it would, and it would make LLVM more popular and more usable in real-world situations.

I don't think that it is a high priority for anyone though.

Then you are disadvantaging the LLVM project and holding it back. I see you are working on a C front-end "clang". Personally I think that a C front-end should be a much lower priority than delivering a COMPLETE backend.

Another C compiler is not needed. GCC and MSVC already do that job, and you won't supplant them. The thing that is really needed is in accordance with the essence of LLVM -- a complete backend solution.

This is highly presumptive of you to say. Your inquiries would be much more well-received if you elided such passages, as they only serve to offend. Likewise, you use “real world situations” as a rather overbroad proxy for your own needs. And you simultaneously use your customers to justify your own ignorance and demean them with the pejorative “Windoze.” It is hypocritical of you to criticize others for their unprofessional responses when your own postings have been and continue to be presumptive, insulting, and poorly researched.

Owen Anderson wrote:

The OP was implying that LLVM is incomplete because it depends on GCC in the backend, which is incorrect. It depends on binutils,

Alright, so I used the wrong name. I said GCC but I should have said binutils. My original point remains valid -- LLVM is an incomplete backend because by itself it is unable to generate a ready-to-execute .EXE or .DLL file.

And binutils is not necessarily available on the customer's computer. Do all Linux and BSD distributions include it preinstalled? Maybe, but what about Windoze and MacOS? I know Windoze does not include it or any equivalent. And Apple's developer tools (Xcode etc) for MacOS X are not preinstalled either, they are a separate download (although perhaps binutils is preinstalled in MacOS X, not sure, haven't checked).

The object writers are just part of the problem (if indeed one exists); a linker is still necessary. That's an area that LLVM is very unlikely to grow into in the near future. Obviously all depends upon contributors, so I cannot state this categorically. But it is a matter of fact that there have been no contributions of native linkers.

So: LLVM provides neither assemblers nor linkers. This is the reality, and no amount of vitriol will not change it. You are not a customer here; there is no vendor-customer relationship. LLVM does what its contributors have seen fit to make it do, neither more nor less. So when it was suggested that you write the code, it was not a flip retort; in fact, such is the only way the project advances.

But I digress. if LLVM provides value to your project, you will need to address the assembler-linker issue in order to use it. Ultimately, this is your responsibility, not ours. However, this is a helpful community (to a point)—so if you are unable to find a solution on your own, we may be able to help. But the community can only provide a constructive response if you give them sufficient context to do so; the first concrete remarks you've made in this direction are to remark how your project already targets C. And you must keep an open mind, since the community will not find you a solution within your framework if you over-constrain the problem.

Given what you've said so far, for instance, we have no way of knowing what constrains you from precompiling your assembly. That is, if you can do compilation at installation time, for the OS X and Windows platforms there's no reason you can't do compilation ahead of time. I think it perfectly reasonable to presume that you already have access to assemblers and linkers in your development environment, and this solves your problem of creating a dependency for customers.

— Gordon

6

Ah! Okay, so "Major Client" said to do it one way and you are not sure how to do that, so you bug us. You first start off by saying that LLVM is an incomplete backend by pointing out that GCC is compleat, yet LLVM still has to use GCC. It's then pointed out to you that you are patently wrong in that regard, and that GCC also produces assembly code, then runs it through an assembler and linker (both of which are separate programs from LLVM). And so because LLVM is now on par with being a "compleat backend" with GCC. You change and say that it's now for customer's computers. And that you can't expect customers to install an assembler and linker on their computers. (Of course, no one was suggesting that you actually do that, but no matter.)

People mentioned a JIT as a possible alternative, but you dismiss that because of some random statement by Major Client.

People say that you should (gasp) actually *build* LLVM, which will give you the DLLs your Major Client so desperately needs, but you won't use the suggested ways they mention, nor will you patch the VC++ project files, nor even submit a bug report. You just bitch and moan and call us "unprofessional", while using inflammatory language which itself is unprofessional.

So, either use the suggested ways to build LLVM, patch the VC++ project files and build LLVM, or submit a *proper* bug report.

-bw

7

No. You don't know. Your Major Client knows. Not you. Your Major Client. I was right to doubt that you knew.

-bw

8 
Emilio Wuerges wrote:
> Is this thread suposed to be a bad joke?

That is a very unprofessional response ...


Terence Parr wrote:
> Yeah, i'm kinda freaked out as i'm sure it's not a joke. 
> i'm in awe. +1 to kill thread before google spiders it.




Again, that is a very unprofessional response.

You may need to invest in a dictionary and look up what the word “professional” means. (Hint: it involves payment. http://dictionary.reference.com/search?q=professional))

The people on this list? Are (mostly) volunteers contributing time and effort to the project. Your demands are not … exactly being professional.

There’s two phrases from various sides of pop culture which could apply to you right now:

  • Show me the code!
  • Show me the money!

Without one or the other of the above, you are contributing nothing but noise and smoke to this project.
9

Ok. There are lots of issues here. As I understand it, LLVM aims to:

. Support Mac OS first, Linux second and Windows is not a priority.

. Provide JIT compilation first and the ability to generate standalone
executables and DLLs second.

. Provide fast compilation first and optimization second.

So LLVM has relatively poor support for Windows, no direct support for DLL
generation and the exact opposite of your performance requirements.

In other words, LLVM does not aspire to do what you are asking and,
consequently, it is not surprising that none of this stuff is implemented.
Therefore, I think it is misguided for you to choose LLVM for your project.

I appreciate that you have customer demands but those demands are very unusual
(and, frankly, absurd!) but you must try to meet them regardless. Nobody here
cares about providing that functionality because nobody else wants it.

Software with the nearest requirements that I can think of is MATLAB, which
bundles precompiled optimized implementations of core numerical algorithms
for a variety of architectures and chooses between them at run-time.

10

Ok. There are lots of issues here. As I understand it, LLVM aims to:

Jon, I'm not sure where you got this list, but I don't find it accurate at all…

. Support Mac OS first, Linux second and Windows is not a priority.
. Provide JIT compilation first and the ability to generate standalone executables and DLLs second.
. Provide fast compilation first and optimization second.

To my estimation:

- Windows is not a priority only to the extent that there are fewer Windows developers using the framework. The *aim* is to be portable, and in fact LLVM does work on Windows. The project files are frequently out of date, though (since most development is done on Unix), so a Windows developer might be more interested in sticking to packaged releases. For comparison, I don't think the Xcode project files can even be used to build the system. :slight_smile:

- Both static and JIT compilation are first-class. The most frequent use of LLVM is probably in Apple's OpenGL stack (JITC) but the most significant driver of development is as the back-end for llvm-gcc and clang (static compilation). That said, JIT support is weaker than static compilation in some areas (exceptions, GC).

- Both speed of compilation and speed of generated code are priorities. This is a dimension of flexibility afforded entirely to the user of the framework, selected by which transformations and analyses they choose to run, and at what time. If anything, however, I would argue that the design is skewed toward more optimal output; LLVM's compile times are never likely to be competitive in a streaming "MB machine code/s" race with low-overhead JIT compilers. I say this because it uses a complex bitcode format and a fairly expensive IR representation (lots of malloc objects). Both of which are explicitly tailored to analysis and low-overhead transformation—not streaming compilation performance. Likewise, there's lots of malloc traffic in the code generator, which (like the IR representation) is a non-optional component.

So I can't agree with the conclusions you draw, except that maybe kr512 might be better served elsewhere :)).

— Gordon