Preferring to use GCC instead of LLVM

Oh another thing, consider this question that some people will be asking: Why not use GCC to do what LLVM does, and skip the hassle of using LLVM entirely?

ESPECIALLY considering that LLVM cannot be used without GCC. Even if you are using LLVM as a back-end only, for compiling LLVM bytecode only, GCC is still required to convert the "llc" output assembly .S file into a ready-to-execute native machine code .EXE or .DLL file.

Thus if LLVM is an incomplete back-end replacement for GCC, why not just use the normal version of GCC?

I do understand the LLVM project and I think it is great and very worthwhile, but while LLVM cannot be used as a complete back-end by itself, then GCC is a more attractive solution for our project.

See, in our project, we have a front-end compiler. For input to the back-end, we can generate LLVM bytecode, or we can generate C source code, and then we need it converted to a native machine code .EXE or DLL ready to load and run. So you see, either GCC or LLVM can be used as our back-end.

Now what advantage does LLVM have that will entice us to pick LLVM over GCC? LLVM is smaller? Not really, because LLVM still requires the use of GCC for the .S file, as I said above.

LLVM would be a considerably more attractive solution than GCC *if* the following were true:
(a) LLVM could be used as a back-end without use of GCC. Meaning LLVM by itself can generate a ready-to-execute .EXE or .DLL file.
(b) LLVM was provided as a precompiled DLL, not only as command-line tools.

Oh another thing, consider this question that some people
will be asking: Why not use GCC to do what LLVM does, and
skip the hassle of using LLVM entirely?

If you'd prefer to use GCC, go for it. No one is forcing you to use LLVM.

ESPECIALLY considering that LLVM cannot be used without GCC.
Even if you are using LLVM as a back-end only, for compiling
LLVM bytecode only, GCC is still required to convert the
"llc" output assembly .S file into a ready-to-execute native
machine code .EXE or .DLL file.

You are seriously ignorant of what LLVM is all about. Please go inform yourself.

-Chris

[snip]

1. Not all language paradigms can be specified in C, hence, you need more raw control.

2. GCC is not the only assembler / linker on windoze. I believe M$ provides assemblers / linkers so people can develop on windoze. At least they did back in the early 80's when I was using DOS. GCC not required :slight_smile:

Ter

Oh another thing, consider this question that some people
will be asking: Why not use GCC to do what LLVM does, and
skip the hassle of using LLVM entirely?

ESPECIALLY considering that LLVM cannot be used without GCC.
Even if you are using LLVM as a back-end only, for compiling
LLVM bytecode only, GCC is still required to convert the
"llc" output assembly .S file into a ready-to-execute native
machine code .EXE or .DLL file.

I don't know about your computer, by mine comes with an assembler. GCC invokes this assembler, and I can invoke it from the command line just fine.

Thus if LLVM is an incomplete back-end replacement for GCC,
why not just use the normal version of GCC?

I do understand the LLVM project and I think it is great and
very worthwhile,

This I agree with.

-bw

Chris Lattner wrote:

If you'd prefer to use GCC, go for it. No one is forcing you to use LLVM.

No, we would prefer to use LLVM, but a missing part in LLVM makes it difficult. It would be wonderful if this missing part could be supplied.

You are seriously ignorant of what LLVM is all about. Please go inform yourself.

Alright, I read some more on llvm.org and it confirmed what I was saying:
http://www.llvm.org/docs/GettingStarted.html#tutorial

See at the end where it says:

Is this thread suposed to be a bad joke?

Yeah, i'm kinda freaked out as i'm sure it's not a joke. i'm in awe.

+1 to kill thread before google spiders it.
Ter

Can you explain why you would like to generate DLLs on the customer's computer
rather than using LLVM as a JIT compiler?

If you are happy with JIT compilation then the article on LLVM in the OCaml
Journal covers everything you need to know to get started.

I think people have already responded to most of your points, but I didn't see this one addressed yet.

kr512 wrote:

Chris Lattner wrote:

However, an own front-end (clang) is in the works, which will remove this dependency.

That doesn't help us. We have a front-end. We need a back-end. But LLVM is an incomplete backend because LLVM by itself cannot generate ready-to-execute .EXE or .DLL file.

If you run "llc -help" you'll find this fragment:

   -filetype - Choose a file type (not all types are supported by all targets):
     =asm - Emit an assembly ('.s') file
     =obj - Emit a native object ('.o') file [experimental]
     =dynlib - Emit a native dynamic library ('.so') file [experimental]

It so happens that because LLVM has a JIT, it already knows how to emit actual executable code instead of textual assembly. However, as the [experimental] indicates -- it doesn't entirely work yet.

Although we can emit the opcodes in binary form, support for the container formats isn't working properly. Thus far, there's an unfinished ELFWriter and MachOWriter in lib/Codegen/. Those could be finished, and PEWriter could be added for Win32 support. No one has done it yet (we just use 'gas' under the hood like gcc does).

If this is blocking you from using LLVM, please consider whether it'd be easier to write this for us than to use something else.

Nick Lewycky

I thought jokes were funny? :wink:

-Chris

You are seriously ignorant of what LLVM is all about.
Please go inform yourself.

Alright, I read some more on llvm.org and it confirmed what
I was saying:
http://www.llvm.org/docs/GettingStarted.html#tutorial

See at the end where it says:
-----------
6.Compile the program to native assembly using the LLC code
generator:
% llc hello.bc -o hello.s
7.Assemble the native assembly language file into a program:
% gcc hello.s -o hello.native
8.Execute the native code program:
% ./hello.native
-----------

See how gcc is invoked to generate the final executable
file. This means LLVM is an incomplete backend,
unfortunately.

That's only a convenience. GCC generates assembly code too and calls the assembler and linker as part of it's execution. You are perfectly able to call the assembler & linker yourself.

Bill Wendling wrote:

I don't know about your computer, by mine comes with an
assembler.

MS Windows does not come with an assembler, AFAIK.

You should really learn how to use Google. Got this as the top hit for "Microsoft Assembler":

-bw

No one ever laughs at my jokes...

-bw

Not that I sympathize with the OP's manners but...

Bill Wendling <isanbard@gmail.com> writes:

See how gcc is invoked to generate the final executable
file. This means LLVM is an incomplete backend,
unfortunately.

That's only a convenience. GCC generates assembly code too and calls
the assembler and linker as part of it's execution. You are perfectly
able to call the assembler & linker yourself.

This means that LLVM requires an assembler and linker. Call it GCC or
binutils, it is irrelevant. The OP point is that LLVM is not a
self-sufficient tool on this aspect.

Of course, if this is a serious problem for the OP, the correct way of
dealing with it is to take constructive, polite actions for correcting
it :slight_smile:

Bill Wendling wrote:

I don't know about your computer, by mine comes with an
assembler.

MS Windows does not come with an assembler, AFAIK.

You should really learn how to use Google. Got this as the top hit for
"Microsoft Assembler":

http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en

This does not *comes* with MS Windows. It is a separate download, which
depends on more downloads (Visual C++ Express Edition) and is for
non-commercial use only. Furthermore, you can't re-distribute it.

OTOH, I'm curious about why the OP needs to produce dlls on the
fly. Isn't the JIT ok?

Not that I sympathize with the OP's manners but...

Bill Wendling <isanbard@gmail.com> writes:

That's only a convenience. GCC generates assembly code too and calls
the assembler and linker as part of it's execution. You are perfectly
able to call the assembler & linker yourself.

This means that LLVM requires an assembler and linker. Call it GCC or
binutils, it is irrelevant. The OP point is that LLVM is not a
self-sufficient tool on this aspect.

Actually, I think Bill's right on target. The OP was implying that LLVM is incomplete because it depends on GCC in the backend, which is incorrect. It depends on binutils, _which GCC also depends on_. Thus LLVM is certainly no less complete than GCC in that sense.

I think the point is that implementing our own linker is definitely beyond the purview of LLVM, and implementing our own object file writer/assembler is honestly pretty low on the priority queue. GCC, the primary alternative, also depends on these things, so having them would be a competitive advantage, not a requirement for parity.

Bill Wendling wrote:

I don't know about your computer, by mine comes with an
assembler.

MS Windows does not come with an assembler, AFAIK.

You should really learn how to use Google. Got this as the top hit for
"Microsoft Assembler":

http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en

This does not *comes* with MS Windows. It is a separate download, which
depends on more downloads (Visual C++ Express Edition) and is for
non-commercial use only. Furthermore, you can't re-distribute it.

From what I read, it's included with copies of Visual Studio. I assume that a paid copy of Visual Studio (as opposed to the free Express edition) includes a version that's licensed for commercial use.

Additionally, LLVM's support for Win32 outside of MinGW/Cygwin is ... spotty at best. And in those environments, the competition is, once again, GCC, which still depends on gas and ld.

The fundamental problem is that both executable formats and linking problems are EXTREMELY platform specific, so it's typically best to leave it up to the system tools. Assembly/object file generation is a _possibility_ for us only because there is some commonality between platforms (i.e. ELF is used on many platforms). But it's still low on the queue because there already system-provided tools that do it better on every platform we care about (*nix + MinGW/Cygwin).

--Owen

P.S. I do agree that, if he has a desperate need to run LLVM outside of MinGW/Cygwin, JIT is his best bet.

Owen Anderson <resistor@mac.com> writes:

That's only a convenience. GCC generates assembly code too and calls
the assembler and linker as part of it's execution. You are perfectly
able to call the assembler & linker yourself.

This means that LLVM requires an assembler and linker. Call it GCC or
binutils, it is irrelevant. The OP point is that LLVM is not a
self-sufficient tool on this aspect.

Actually, I think Bill's right on target. The OP was implying that
LLVM is incomplete because it depends on GCC in the backend, which is
incorrect. It depends on binutils, _which GCC also depends on_. Thus
LLVM is certainly no less complete than GCC in that sense.

There is a culture schism here. The Windows folks are accustomed to
monolithic packages that does all the work. For Unix folks, having
"tool-chains" is the natural way. For a Windows folk, having extra
requirements goes against what he is accustomed to. If LLVM is
perceived as a backend, but it fails short into producing the final
result, the typical Windows folk will be disappointed.

The OP talks about GCC because that's what he sees, irrespectively of
the other tools behind. In Windows, installing GCC implies installing
binutils, either via MinGW or Cygwin. This is deeply rooted into the
Windows development culture: a "compiler" has all needed parts for
creating an executable. They talk about the Visual C++ compiler, or the
Borland compiler, never about the Borland compiler and linker.

I think the point is that implementing our own linker is definitely
beyond the purview of LLVM, and implementing our own object file
writer/assembler is honestly pretty low on the priority queue. GCC,
the primary alternative, also depends on these things, so having them
would be a competitive advantage, not a requirement for parity.

Absolutely agreed. I'm just trying to explain from where the OP (seems
to) come.

"Microsoft Assembler":

http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en

This does not *comes* with MS Windows. It is a separate download,
which
depends on more downloads (Visual C++ Express Edition) and is for
non-commercial use only. Furthermore, you can't re-distribute it.

From what I read, it's included with copies of Visual Studio. I
assume that a paid copy of Visual Studio (as opposed to the free
Express edition) includes a version that's licensed for commercial
use.

Exactly. And this is very different from the GNU/Linux and BSD case,
where the existence of a compiler, assembler and linker along the OS is
considered a "reasonable requirement" and part of the standard
installation in most cases. If the OP plans to deploy his compiler on
Windows, he can't assume the existence of an assembler/linker on most
user's machines. He becomes dependent of third-party packages, hence
LLVM is not a "complete backend" from his (limited) point of
view. (Leaving aside the fact that a compiler backend is not required to
generate executable files, but his insistence on this point is related
to the "monolithic package" paradigm I mentioned above).

Additionally, LLVM's support for Win32 outside of MinGW/Cygwin is
... spotty at best.

Sure. And, AFAIK, even if you want to use MASM, you need to add support
for his assembler format to LLVM.

And in those environments, the competition is, once again, GCC, which
still depends on gas and ld.

The fundamental problem is that both executable formats and linking
problems are EXTREMELY platform specific,

[snip]

Agreed. I'm just trying to explain that, from the POV of the OP's
culture, the demands he expressed are quite reasonable. This narrow view
of the world and being... ummmh... too demanding, are the only problems
with the OP message :slight_smile:

Expect more messages like this as LLVM becomes more popular outside
professional and academical circles.

FWIW, I wouldn't be surprised if the LLVM project eventually grew more of native toolchain support (e.g. assembler, linker, etc). It would fit naturally with the growing scope of the project. I don't think that it is a high priority for anyone though.

One advantage of having at least simple assembler support is being able to handle inline asm in the (currently incomplete and buggy) macho/elf writers.

-Chris

I tried to compile on Windows ( using the win32/llvm.sln in MSVC Express 2008 ) the new 2.3 branch and here are the results:

1. I needed to copy the file Configure.exe.embed.manifest to Configure.exe.intermediate.manifest . This is a bug from the previous llvm release (2.2)

2. I copied the file SimplifyLibCalls.cpp from lib/Transforms/Scalar to lib/Transforms/IPO. I think the position of this file was updated but the MSVC project wasn't.

3. I put a "return SDOperand();" on X86ISelLowering.cpp, on function "SDOperand X86TargetLowering::LowerVAARG(SDOperand Op, SelectionDAG &DAG)" after "abort()", to make compiler happy to compile it.

4. I needed to give "Build Solution" more times, because there are some incorrect dependencies between projects and some of them expect results from the future builded ones.

After over 4500 warnings, I think most of them because of the automatically generated code, I succeded to compile almost all the projects, except opt and bugpoint. At these two I had an error:

error LNK2019: unresolved external symbol "class llvm::FunctionPass * __cdecl llvm::createLibCallAliasAnalysisPass(class llvm::LibCallInfo *)" (?createLibCallAliasAnalysisPass@llvm@@YAPAVFunctionPass@1@PAVLibCallInfo@1@@Z) referenced in function "public: __thiscall `anonymous namespace'::ForcePassLinking::ForcePassLinking(void)" (??0ForcePassLinking@?A0xd1580448@@QAE@XZ)

which I wasn't able to fix it.

I hope these will be fixed on the final 2.3 release so it can be compiled out of the box on MSVC Express 2008.

Now I'm checking on some Windows assemblers and other tools which can complete llvm to make it a full backend (with the final result an exe or dll file).

Razvan

PS: I also tried to compile llvm 2.3 in MSYS using the latest MinGW with gcc 4.3.0. This wasn't able to compile llvm at all because some errors like:

d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h: In member function 'bool __gnu_cxx::_EqualOpConcept<_First, _Second>::__constraints_() [with _First = llvm::BasicBlock, _Second = llvm::BasicBlock*]':
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:296: instantiated from 'void __gnu_cxx::_EqualOpConcept<_First, _Second>::__constraints() [with _First = llvm::BasicBlock, _Second = llvm::BasicBlock*]'
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:62: instantiated from 'void __gnu_cxx::__function_requires() [with _Concept = __gnu_cxx::_EqualOpConcept<llvm::BasicBlock, llvm::BasicBlock*>]'
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/stl_algo.h:3810: instantiated from '_IIter std::find(_IIter, _IIter, const _Tp&) [with _IIter = llvm::PredIterator<llvm::BasicBlock, llvm::value_use_iterator<llvm::User> >, _Tp = llvm::BasicBlock*]'
BasicBlock.cpp:180: instantiated from here
d:\msys\bin\../lib/gcc/i386-pc-mingw32/4.3.0/include/c++/bits/boost_concept_check.h:296: error: no match for 'operator==' in '((__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, llvm::BasicBlock*>*)this)->__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, llvm::BasicBlock*>::__a == ((__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, llvm::BasicBlock*>*)this)->__gnu_cxx::_EqualOpConcept<llvm::BasicBlock, llvm::BasicBlock*>::__b'
D:/msys/home/Razvan/release_23/include/llvm/ADT/APInt.h:1242: note: candidates are: bool llvm::operator==(uint64_t, const llvm::APInt&)
make[1]: *** [/home/Razvan/release_23/lib/VMCore/Release/BasicBlock.o] Error 1

Not that I sympathize with the OP's manners but...

Bill Wendling <isanbard@gmail.com> writes:

See how gcc is invoked to generate the final executable
file. This means LLVM is an incomplete backend,
unfortunately.

That's only a convenience. GCC generates assembly code too and calls
the assembler and linker as part of it's execution. You are perfectly
able to call the assembler & linker yourself.

This means that LLVM requires an assembler and linker. Call it GCC or
binutils, it is irrelevant. The OP point is that LLVM is not a
self-sufficient tool on this aspect.

Then the exact same thing is true for GCC.

Of course, if this is a serious problem for the OP, the correct way of
dealing with it is to take constructive, polite actions for correcting
it :slight_smile:

Bill Wendling wrote:

I don't know about your computer, by mine comes with an
assembler.

MS Windows does not come with an assembler, AFAIK.

You should really learn how to use Google. Got this as the top hit for
"Microsoft Assembler":

http://www.microsoft.com/downloads/details.aspx?FamilyID=7a1c9da0-0510-44a2-b042-7ef370530c64&displaylang=en

This does not *comes* with MS Windows. It is a separate download, which
depends on more downloads (Visual C++ Express Edition) and is for
non-commercial use only. Furthermore, you can't re-distribute it.

It's a separate download for me too. And there are other tools available for MS Windows which achieve the same goals. The point is that he was wrong.

OTOH, I'm curious about why the OP needs to produce dlls on the
fly. Isn't the JIT ok?

I doubt he knows.

-bw

kr512 wrote:

Oh another thing, consider this question that some people
will be asking: Why not use GCC to do what LLVM does, and
skip the hassle of using LLVM entirely?

ESPECIALLY considering that LLVM cannot be used without GCC.
Even if you are using LLVM as a back-end only, for compiling
LLVM bytecode only, GCC is still required to convert the
"llc" output assembly .S file into a ready-to-execute native
machine code .EXE or .DLL file.

Thus if LLVM is an incomplete back-end replacement for GCC,
why not just use the normal version of GCC?

I do understand the LLVM project and I think it is great and
very worthwhile, but while LLVM cannot be used as a complete
back-end by itself, then GCC is a more attractive solution
for our project.

See, in our project, we have a front-end compiler. For
input to the back-end, we can generate LLVM bytecode, or we
can generate C source code, and then we need it converted to
a native machine code .EXE or DLL ready to load and run. So
you see, either GCC or LLVM can be used as our back-end.

Now what advantage does LLVM have that will entice us to
pick LLVM over GCC? LLVM is smaller? Not really, because
LLVM still requires the use of GCC for the .S file, as I
said above.

LLVM would be a considerably more attractive solution than
GCC *if* the following were true:
(a) LLVM could be used as a back-end without use of GCC.
Meaning LLVM by itself can generate a ready-to-execute .EXE
or .DLL file.
  

As other have stated, most systems come with an assembler (even GCC relies on an assembler to generate native code). While I agree that generating ELF object files would be a nice feature, the presence of an assembler on all supported platforms has made this a lower priority, so (as far as I know), no one has done it.

If people are interested in finishing the work that has already been started on it, we gratefully accept patches.

(b) LLVM was provided as a precompiled DLL, not only as
command-line tools.
  

LLVM is actually designed as a set of libraries with a (in my humble opinion) relatively easy to use API. The command line tools are only wrappers, so to speak, around these libraries. You can write your own program that directly links in and uses the LLVM libraries to perform code generation, program analysis, optimization, and even JIT compiling. Your LLVM tools do not need to pass input through the opt or llc programs; you can link and use the libraries that opt and llc use to do their jobs.

About the only thing we don't plan to do (as far as I know) is precompile the LLVM libraries. We prefer source distributions (because LLVM supports many different OS/hardware combinations, and LLVM is easy to compile on most platforms).

For more information on the LLVM APIs, I'd refer you to the following:

Programmer's Manual:
http://llvm.org/docs/ProgrammersManual.html

LLVM Tutorial:
http://llvm.org/docs/tutorial/

LLVM Libraries (out of date, but gives you an idea of how LLVM's structure supports DLLs):
http://llvm.org/docs/UsingLibraries.html

Doxygen (auto-generated information on the LLVM classes and libraries):
http://llvm.org/doxygen/namespaces.html

Other documentation can be found at http://llvm.org/docs.

-- John T.

I tried to compile on Windows ( using the win32/llvm.sln in MSVC Express
2008 ) the new 2.3 branch and here are the results:

It would be great to get any windows improvements into the 2.3 release if possible.

1. I needed to copy the file Configure.exe.embed.manifest to
Configure.exe.intermediate.manifest . This is a bug from the previous llvm
release (2.2)

Please send a patch.

2. I copied the file SimplifyLibCalls.cpp from lib/Transforms/Scalar to
lib/Transforms/IPO. I think the position of this file was updated but the
MSVC project wasn't.

Please update the project file and send a patch for it.

3. I put a "return SDOperand();" on X86ISelLowering.cpp, on function
"SDOperand X86TargetLowering::LowerVAARG(SDOperand Op, SelectionDAG &DAG)"
after "abort()", to make compiler happy to compile it.

This went into mainline, Tanya, can we pull this patch into 2.3?
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20080512/062268.html

4. I needed to give "Build Solution" more times, because there are some
incorrect dependencies between projects and some of them expect results from
the future builded ones.

Please fix them :slight_smile:

After over 4500 warnings, I think most of them because of the automatically
generated code, I succeded to compile almost all the projects, except opt
and bugpoint. At these two I had an error:

error LNK2019: unresolved external symbol "class llvm::FunctionPass *
__cdecl llvm::createLibCallAliasAnalysisPass(class llvm::LibCallInfo *)"
(?createLibCallAliasAnalysisPass@llvm@@YAPAVFunctionPass@1@PAVLibCallInfo@1@@Z)
referenced in function "public: __thiscall `anonymous
namespace'::ForcePassLinking::ForcePassLinking(void)"
(??0ForcePassLinking@?A0xd1580448@@QAE@XZ)

which I wasn't able to fix it.

This sounds like lib/Analysis/LibCallAliasAnalysis.cpp needs to be added to the project file.

I hope these will be fixed on the final 2.3 release so it can be compiled
out of the box on MSVC Express 2008.

We would love to have this fixed, but since most of the core developers aren't using VC++, the only way it will get fixed for LLVM 2.3 is if someone sends up patches. Can you please do this?

-Chris