Newby questions

[Hartmut_Kaiser]

Hi all,

I'm new to this list and this project, which IMHO is very important for the
whole C++ community. I have a couple of questions - please excuse my
ignorance if these have been answered before.

1. I couldn't find any build instructions/project files to build clang using
MSVC8, so I decided to create these. I tried to use similar build settings
as have been used in the corresponding MSVC project files for LLVM. Almost
everything compiled fine, except for a bigger issue and some really minor
platform specific issues, which I have been able to address. The bigger
issue is that the clang::FileEntry class retrieves the size of the file
using stat, but because of the EOL translation done on Windows the file
usually is smaller in memory than it is on disk. I introduced a hack in the
ReadFileFast() function to circumvent an infinite loop there, but this has
other effects I'm not able to resolve ad hoc. How is this solved in LLVM?

2. The clang driver builds and runs for me now (after this hack) but I'm
getting an assertion when I started to run the array.c test in the
test/CodeGen directory: Assertion failed: isa<X>(Val) && "cast<Ty>()
argument of incompatible type!", file
c:\CVS\llvm\include\llvm/Support/Casting.h, line 199. Is this a known issue
or did something go wrong during my compilation, or is this related to the
issue I described above?

3. After getting these assertions I moved to the test/Lexer directory to
see, if the preprocessor is working. The tests there seem to pass. So I
started to feed the files from the Boost.Wave testsuite to clang in pp mode
(clang -E). Unfortunately many of these tests fail (produce wrongly
preprocessed output). Is this a known issue? Would there be any interest in
using Boost.Wave as the preprocessor/lexer for clang?

4. Is there any interest in getting the MSVC8 project files mentioned above?
If yes, what's the suggested procedure?

Thanks!
Regards Hartmut

[Chris_Lattner]

Hi all,

I'm new to this list and this project, which IMHO is very important for the
whole C++ community. I have a couple of questions - please excuse my
ignorance if these have been answered before.

Welcome!

1. I couldn't find any build instructions/project files to build clang using
MSVC8, so I decided to create these. I tried to use similar build settings
as have been used in the corresponding MSVC project files for LLVM. Almost
everything compiled fine, except for a bigger issue and some really minor
platform specific issues, which I have been able to address. The bigger
issue is that the clang::FileEntry class retrieves the size of the file
using stat, but because of the EOL translation done on Windows the file
usually is smaller in memory than it is on disk. I introduced a hack in the
ReadFileFast() function to circumvent an infinite loop there, but this has
other effects I'm not able to resolve ad hoc. How is this solved in LLVM?

LLVM has a system abstraction library, but it needs to be properly extended to handle this case. Can you please send in the patch for for the change?

2. The clang driver builds and runs for me now (after this hack) but I'm
getting an assertion when I started to run the array.c test in the
test/CodeGen directory: Assertion failed: isa<X>(Val) && "cast<Ty>()
argument of incompatible type!", file
c:\CVS\llvm\include\llvm/Support/Casting.h, line 199. Is this a known issue
or did something go wrong during my compilation, or is this related to the
issue I described above?

I'm away from my computer, so I can't check right now. It is possible that it is a real regression, but we also haven't done much testing on windows, so it may be a platform issue.

3. After getting these assertions I moved to the test/Lexer directory to
see, if the preprocessor is working. The tests there seem to pass. So I
started to feed the files from the Boost.Wave testsuite to clang in pp mode
(clang -E).

Ok, cool.

Unfortunately many of these tests fail (produce wrongly
preprocessed output). Is this a known issue?

Can you give an example? When producing a .i file, clang futzes with whitespace a bit to make sure the output will relex into the proper token stream (this only affects .i file writing, not the parser). You may be hitting this or it may be a real bug. If it is a bug, we'll fix it

Would there be any interest in
using Boost.Wave as the preprocessor/lexer for clang?

I'm not emotionally attached to our current preprocessor - if wave is as performant and feature compatible with clangs, we can switch.

4. Is there any interest in getting the MSVC8 project files mentioned above?
If yes, what's the suggested procedure?

Yes! Please send them to the cfe-commits list.

-Chris

[_Neil_Booth]

Hartmut Kaiser wrote:-

1. I couldn't find any build instructions/project files to build clang using
MSVC8, so I decided to create these. I tried to use similar build settings
as have been used in the corresponding MSVC project files for LLVM. Almost
everything compiled fine, except for a bigger issue and some really minor
platform specific issues, which I have been able to address. The bigger
issue is that the clang::FileEntry class retrieves the size of the file
using stat, but because of the EOL translation done on Windows the file
usually is smaller in memory than it is on disk. I introduced a hack in the
ReadFileFast() function to circumvent an infinite loop there, but this has
other effects I'm not able to resolve ad hoc. How is this solved in LLVM?

I think we should be reading the files in binary mode.

3. After getting these assertions I moved to the test/Lexer directory to
see, if the preprocessor is working. The tests there seem to pass. So I
started to feed the files from the Boost.Wave testsuite to clang in pp mode
(clang -E). Unfortunately many of these tests fail (produce wrongly
preprocessed output). Is this a known issue? Would there be any interest in
using Boost.Wave as the preprocessor/lexer for clang?

I doubt it; Chris wrote it for speed and for the interface it provides
the front end. We should just fix whatever the failures are. Stuff
like UCNs aren't yet implemented.

Just my opinions above.

Neil.

[Hartmut_Kaiser]

Neil Booth wrote:

> 1. I couldn't find any build instructions/project files to
build clang
> using MSVC8, so I decided to create these. I tried to use similar
> build settings as have been used in the corresponding MSVC project
> files for LLVM. Almost everything compiled fine, except for
a bigger
> issue and some really minor platform specific issues, which I have
> been able to address. The bigger issue is that the clang::FileEntry
> class retrieves the size of the file using stat, but because of the
> EOL translation done on Windows the file usually is smaller
in memory
> than it is on disk. I introduced a hack in the
> ReadFileFast() function to circumvent an infinite loop
there, but this
> has other effects I'm not able to resolve ad hoc. How is
this solved in LLVM?

I think we should be reading the files in binary mode.

Ok, this did the trick. If you tell me, how I detect compilation on a
windows platform in LLVM/clang style (i.e. which pp constant to use) I'll
submit a patch.

> 3. After getting these assertions I moved to the test/Lexer
directory
> to see, if the preprocessor is working. The tests there
seem to pass.
> So I started to feed the files from the Boost.Wave
testsuite to clang
> in pp mode (clang -E). Unfortunately many of these tests
fail (produce
> wrongly preprocessed output). Is this a known issue? Would there be
> any interest in using Boost.Wave as the preprocessor/lexer
for clang?

I doubt it; Chris wrote it for speed and for the interface it
provides the front end. We should just fix whatever the
failures are. Stuff like UCNs aren't yet implemented.

The errors I've seen were because of the problem above. Now everything I
checked (especially wrt macro expansion order and macro parameter
evaluation) looks ok. Good job! There aren't too much preprocessors around
being able to get all the Wave test cases right (so far, actually besides
Wave itself there was only 'gcc -E' doing it correctly).

Regards Hartmut

[Hartmut_Kaiser]

Chris Lattner wrote:

> 1. I couldn't find any build instructions/project files to build
> clang using
> MSVC8, so I decided to create these. I tried to use similar build
> settings
> as have been used in the corresponding MSVC project files
for LLVM.
> Almost
> everything compiled fine, except for a bigger issue and
some really
> minor
> platform specific issues, which I have been able to address. The
> bigger
> issue is that the clang::FileEntry class retrieves the size of the
> file
> using stat, but because of the EOL translation done on Windows the
> file
> usually is smaller in memory than it is on disk. I
introduced a hack
> in the
> ReadFileFast() function to circumvent an infinite loop there, but
> this has
> other effects I'm not able to resolve ad hoc. How is this
solved in
> LLVM?

LLVM has a system abstraction library, but it needs to be properly
extended to handle this case. Can you please send in the patch for
for the change?

The O_BINARY flag did the trick (see my other mail). I'll submit a patch for
the other changes.

> 2. The clang driver builds and runs for me now (after this
hack) but
> I'm
> getting an assertion when I started to run the array.c test in the
> test/CodeGen directory: Assertion failed: isa<X>(Val) && "cast<Ty>()
> argument of incompatible type!", file
> c:\CVS\llvm\include\llvm/Support/Casting.h, line 199. Is this a
> known issue
> or did something go wrong during my compilation, or is this
related
> to the
> issue I described above?

I'm away from my computer, so I can't check right now. It is
possible
that it is a real regression, but we also haven't done much
testing on
windows, so it may be a platform issue.

Ok.

> Unfortunately many of these tests fail (produce wrongly
> preprocessed output). Is this a known issue?

Can you give an example? When producing a .i file, clang
futzes with
whitespace a bit to make sure the output will relex into the proper
token stream (this only affects .i file writing, not the
parser). You
may be hitting this or it may be a real bug. If it is a bug, we'll
fix it

Fixed with the O_BINARY patch. Sorry for the noise.

> 4. Is there any interest in getting the MSVC8 project files
> mentioned above?
> If yes, what's the suggested procedure?

Yes! Please send them to the cfe-commits list.

Will do.

Regards Hartmut

[Chris_Lattner]

Neil Booth wrote:

1. I couldn't find any build instructions/project files to

build clang

using MSVC8, so I decided to create these. I tried to use similar
build settings as have been used in the corresponding MSVC project
files for LLVM. Almost everything compiled fine, except for

a bigger

issue and some really minor platform specific issues, which I have
been able to address. The bigger issue is that the clang::FileEntry
class retrieves the size of the file using stat, but because of the
EOL translation done on Windows the file usually is smaller

in memory

than it is on disk. I introduced a hack in the
ReadFileFast() function to circumvent an infinite loop

there, but this

has other effects I'm not able to resolve ad hoc. How is

this solved in LLVM?

I think we should be reading the files in binary mode.

Ok, this did the trick. If you tell me, how I detect compilation on a
windows platform in LLVM/clang style (i.e. which pp constant to use) I'll
submit a patch.

I'm behind and not following closely, did the patch to open files in binary mode make it into the tree?

The errors I've seen were because of the problem above. Now everything I
checked (especially wrt macro expansion order and macro parameter
evaluation) looks ok. Good job! There aren't too much preprocessors around
being able to get all the Wave test cases right (so far, actually besides
Wave itself there was only 'gcc -E' doing it correctly).

Nice! We have a bunch of nasty testcases of our own in clang/test/Preprocessor if you're interested.

-Chris

[Hartmut_Kaiser]

Chris,

>> I think we should be reading the files in binary mode.
>
> Ok, this did the trick. If you tell me, how I detect
compilation on a
> windows platform in LLVM/clang style (i.e. which pp constant to
> use) I'll
> submit a patch.

I'm behind and not following closely, did the patch to open
files in binary mode make it into the tree?

It's in the SVN. The only thing I'm not sure with is the condition it is
guarded by. Currently it's:

#if defined(_WIN32) || defined(_WIN64)
Windows specifics here
#else
...
#endif

I just don't know, what's pp constant is usually used in Clang for this kind
of stuff. Please advise.

> The errors I've seen were because of the problem above. Now
everything
> I checked (especially wrt macro expansion order and macro parameter
> evaluation) looks ok. Good job! There aren't too much preprocessors
> around being able to get all the Wave test cases right (so far,
> actually besides Wave itself there was only 'gcc -E' doing it
> correctly).

Nice! We have a bunch of nasty testcases of our own in
clang/test/ Preprocessor if you're interested.

Ohh! Cool. Found a bug in wave using these :-P! May I include the
corresponding test to the Wave test suite in Boost?

Regards Hartmut

[Anton_Korobeynikov]

Hello, Harmut.

#if defined(_WIN32) || defined(_WIN64)
Windows specifics here
#else
..
#endif

Please don't forget about mingw32 It's also running on windows.

[Hartmut_Kaiser]

> #if defined(_WIN32) || defined(_WIN64) Windows specifics
here #else ..
> #endif
Please don't forget about mingw32 It's also running on windows.

So what's the general approach to this kind of pp magic? Is there some
existing pp constant I could use?
And BTW, is there some config file for Clang?

Regards Hartmut

[Anton_Korobeynikov]

Hello, Hartmut

So what's the general approach to this kind of pp magic? Is there some
existing pp constant I could use?

I don't exactly know, how the things are going in clang. But I'd really
prefer not to use any platform-dependent defines in the "common"
sources. Maybe some indirection via configuration defines (like
"CLANG_ON_WINDOWS" - same as "LLVM_ON_UNIX" / "LLVM_ON_WINDOWS" and
libSystem / libSupport) should be introduced.

[Chris_Lattner]

clang gets access to all the llvm headers, so you should be able to do:

#include "llvm/Config/config.h"

#ifdef LLVM_ON_WIN32
...
#endif

Does that work?

-Chris

[Chris_Lattner]

Hartmut,

does #ifdef LLVM_ON_WIN32 work after #include "llvm/Config/config.h" ?

-Chris

[Chris_Lattner]

Sure, go for it.

-Chris

[Hartmut_Kaiser]

Chris,

>>> #if defined(_WIN32) || defined(_WIN64) Windows specifics
>> here #else ..
>>> #endif
>> Please don't forget about mingw32 It's also running on windows.
>
> So what's the general approach to this kind of pp magic? Is
there some
> existing pp constant I could use?
> And BTW, is there some config file for Clang?

does #ifdef LLVM_ON_WIN32 work after #include "llvm/Config/config.h" ?

Sure, this does work. Thanks.

Regards Hartmut

[Chris_Lattner]

Nice, thanks!

-Chris