accessing a bitcode library exported from C++ using the JIT

Hello,

My partner and I am making a small app needs to access a C++ library from the LLVM 2.5 JIT. We've made sure that there are no classes and have put 'extern "c"' in front of the functions we need to access. In order to make this work, we seem to need to have a bitcode version of libstdc++ so we can avoid writing our own implementations of std::string and std::vector. I've been trying to get the version of libstdc++ that came with llvm-gcc-4.2-2.5 to configure and compile. I've set the environment variables to use llvm-gcc and llvm-g++ respectively and placed the --emit-llvm flag in the CCFLAGS and CXXFLAGS environment variables. Configure works but make fails with:

from /Users/samuraicrow/Documents/llvm-gcc4.2-2.5.source/libstdc++-v3/include/precompiled/extc++.h:60:
/Users/samuraicrow/Documents/llvm-gcc4.2-2.5.source/libstdc++-v3/include/ext/pb_ds/detail/priority_queue_base_dispatch.hpp:79: error: ISO C++ forbids declaration of ‘rc_binomial_heap_’ with no type
/Users/samuraicrow/Documents/llvm-gcc4.2-2.5.source/libstdc++-v3/include/ext/pb_ds/detail/priority_queue_base_dispatch.hpp:79: error: expected ‘;’ before ‘<’ token

I'm building on MacOSX 10.5.8 and my parnter will be building on MinGW under Windows XP.

Is this a problem with the source or with my configuration? Has this been solved in version 2.6 prerelease or ToT?

Thanks in advace for your help,

--Sam

It's probably a problem with your configuration, since llvm-gcc can
definitely build libstdc++ as part of the normal build.

This seems like the wrong approach, though... can't you just make your
C APIs take char*/T* instead?

-Eli

Hello Eli,

We're using the LLVM Value * class functions to box and unbox values and functions for our stack. The stack needs to be able to take indexing without changing the stack pointer so we're using a std::vector for that. The std::string section would be a lot easier to replace than the two I just mentioned since we store the length separately.

This library is serving as part of an interpreted parser generator so it needs to be able to handle all types and functions that we choose to feed into it. For now it is i32, Float and String and in the future, Doubles and functions of various sorts need to go onto the stack. The only thing I doubt about our approach is that we might be able to get away with using an interpreter instead of a JIT since the code is fed through the LLVM assembly parser just as it gets executed.

If you're wondering why we're doing an interpreted PEG parser generator rather than Boost Spirit 2.x, it's because we need it to be easier to debug the parser. Once the parser is debugged it can be fed into a compiled parser generator and "frozen" into stand-alone parser code.

--Sam

If you really want to stick with std::string and std::vector, I'd
suggest just writing some wrappers for the relevant functions. For
example:

extern "C" char* String_cstr(std::string*);
extern "C" void Stack_pushback(std::vector*, void* value);

You can even compile these to bitcode with llvm-g++ and inline them
into your other IR.

I don't think compiling libstdc++ to bitcode actually helps you here;
you can call the relevant functions in libstdc++ whether or not
they're bitcode.

-Eli

Hi Eli,

That's exactly what we are trying to do. All of the parameters are passed in as text and parsed using StringStreams into their relavent types in the Value stack. You can view the current code at http://mattathias.cvs.sourceforge.net/viewvc/mattathias/mattathias/modules/front_ends/Amos/BasicLib.cpp?view=markup and the BasicLib.h included in it only has a bunch of LLVM includes and an irrelevant previous attempt at code abstraction and a C struct to return the length-terminated string in. I think my mistake is that I'm using Value * as a return code in one place instead of a void *. I think that's causing the name mangling to kick in. I'll see what I can do to fix it.

Thanks for talking me through the problem,

--Sam

You do know that Spirit2.1 has very nice debugging facilities,
everything from error handlers so you can handle parsing failures to
getting detailed info reports on the entire processing process (of
which you can do something like dump it to cout or something and you
get this nice tree of everything that is tried, what passes, what
fails, etc...). It is exceedingly detailed, so I am not sure what
your homegrown one would have over it, if you have an example then I
could add that functionality to Boost.Spirit2.1.

Hello OvermindDL1,

We are implementing an extensible language. That's one where you can add commands and constructs to the language without having to recompile the parser. We want compilation of the parser in order to "freeze" it but only as an option. One goal is to eventually get the macro functions of our language to the point where they are equivalent to the template functions of C++ so we can make a self-hosting langauge.

Also, all of the action nodes in our example PEG parser are written in LLVM Assembly rather than only C++ (although they call functions from a C++ library) so I doubt it would work in Spirit2.1 without writing a syntax converter. The syntax converter may come in time but it's only optional. If you're really interested in a partially finished example parser, it is at http://mattathias.cvs.sourceforge.net/viewvc/mattathias/mattathias/modules/front_ends/Amos/ . It contains a PEG parser file and the source to a C++ library included in that PEG file.

Thanks for your interest,

--Sam

We are implementing an extensible language. That's one where you can add commands and constructs to the language without having to recompile the parser. We want compilation of the parser in order to "freeze" it but only as an option. One goal is to eventually get the macro functions of our language to the point where they are equivalent to the template functions of C++ so we can make a self-hosting langauge.

Ah, I have been doing the same thing, letting read-macro's and macro's
(ala lisp/scheme style), but I created a new terminal in Spirit that
can do the callbacks, but yea, I intend to re-implement the PEG parser
as read-macro's in my language and re-implement the whole spirit
parser in there, intending for the whole thing to become self-hosting
in LLVM by the end. :slight_smile:

Also, all of the action nodes in our example PEG parser are written in LLVM Assembly rather than only C++ (although they call functions from a C++ library) so I doubt it would work in Spirit2.1 without writing a syntax converter. The syntax converter may come in time but it's only optional. If you're really interested in a partially finished example parser, it is at http://mattathias.cvs.sourceforge.net/viewvc/mattathias/mattathias/modules/front_ends/Amos/ . It contains a PEG parser file and the source to a C++ library included in that PEG file.

Actually, creating a new terminal (as I did) in Spirit 2.1 is
extremely easy now (compared to previous spirit's, 2.0, 1.x), that is
how I handled it, but yea, as stated, I intend to eventually move to
to completely self-hosted LLVM code, have the interpreter interpret
itself.

I will take a look at your link later though, curious as to what you
are making. :slight_smile: