Clang Configuration Manager

Hi,

I want to work on Configuration Manager open project.

In the description of the project (here) the first step would be to build a CFLAGS file. I think it is something similar to the Makefile in the root directory of the Clang, is that right?

Then, I do not understand very well, how this project would be different from what CMake does?

And the last, for now, would Python be a suitable choice for this project or better write it in C++?

Thank you very much for your time!

I want to work on Configuration Manager open project.

Hi Cornel,

I wrote that spec, so I'll try to give you an overview of what I
meant. That doesn't mean you have to do exactly that, just to guide
you on your implementation.

In the description of the project (here) the first step would be to build a
CFLAGS file. I think it is something similar to the Makefile in the root
directory of the Clang, is that right?

No. The idea was to do something similar to "llvm-config", but the
other way around.

The llvm-config tool, like similar ones, gives you a way to collect
the flags needed to compile an LLVM based project. What this first
step would be is to do the same for the hardware.

GCC's structure is largely based on configure-time hard-coded values.
So, from the same source, you can build a cross-compiler from
virtually any host to any target, and the defaults and options will be
restricted to the target arch. LLVM doesn't work like that.

When you build LLVM, you build everything. You can cross-compile from
one host to any target (if you compile all back-ends) on the same
binary. Because of that, some options get slightly different
behaviour, or they're only valid for one or another back-end, or they
need another options to be set and correct, otherwise things blow up
badly.

So, there are two side of stage 1:

1. Identify the host architecture (system introspection), and present
the best combination of flags (mcpu, mfpu, march, etc) to produce the
fastest code on the *host*. This will detect the CPU, SIMD,
co-processors, variations and choose the right options.

2. Identify the target, when host != target. This is more complicated,
since however you detect it will depend on the target. If the target
has an OS, you can telnet/ssh into it and query the system, but some
OSs will need a server running on the target (Windows?). If the target
is bare-metal, you'll need some JTAG equivalent, and thus interface
with drivers on the host system.

This second part would be more of a large collection of tools, rather
than one single script.

The second stage is to put these scripts into a local database (a JSON
file would be enough), so that you could choose via Clang
-mconfig=target-name and it'd automatically detect and pull the config
from inside Clang. The scripts would still be used to create the
database, though.

And the last, for now, would Python be a suitable choice for this project or
better write it in C++?

Python would be perfect.

cheers,
--renato

Hi Renato,

Thank you very much for your help!

If I understood correctly, the first thing to implement is a tool that gathers information about the target, like (you mentioned):

  • cpu, OS, etc.

The information gathered at this step will be used to generate the JSON file / local database. Did I understood correctly?

On the step when target != host, I need to think and research more, because I do not have a clear image.

Thank you!

În Mar, 13 oct. 2015 la 13:06, Renato Golin <renato.golin@linaro.org> a scris:

If I understood correctly, the first thing to implement is a tool that
gathers information about the target, like (you mentioned):
- cpu, OS, etc.
The information gathered at this step will be used to generate the JSON file
/ local database. Did I understood correctly?

Yes, when target == host. A simple Python script that knows how to
gather that information (Clang knows how to, you can either use a
similar logic or expose that as an API).

On the step when target != host, I need to think and research more, because
I do not have a clear image.

That's the hard part. :slight_smile:

A simpler way to do this is to keep the database in git.

Let's say you wrote a script that gathers all needed config for Linux
on x86, ARM, MIPS and PPC. They should all be *very* similar.

   Step #1: Generate CFLAGS from that info

You can use it like llvm-config, ex:

$ clang++ -c foo.c `target-info --target=host`

This would get the host's config and output CFLAGS, which will be used
by the Clang command line.

   Step #2: Try remote execution (optional)

Assuming you have ssh access to an "arm-board":

$ clang++ -c foo.cpp `target-info --target=foo@arm-board.localdomain`

That would supposedly connect to the target running Linux and run the
same script, producing CFLAGS.

   Step #3: Generate a JSON file

x86$ target-info --name="i7" --target=host >> targets.json
panda$ target-info --name=pandaboard" --target=host >> targets.json
mips64$ target-info --name="mymips" --target=host >> targets.json

Concatenating all results into a JSON file, and then using that file,
say, in your home dir, default to something like ".llvm-targets.json"

$ cp targets.json .llvm-targets.json

$ clang++ -c foo.cpp `target-info --target=pandaboard`

This would look for the JSON file, find the "pandaboard" entry, and
then output the CFLAGS.

   Step #4: Clang integration

Once the JSON location is stabilised, make Clang read the file and set
internally the flags (no CFLAGS)

$ clang++ -target=pandaboard -c foo.cpp

Target will not be recognised by the front-end, then your module kicks
in and finds the .llvm-targets.json, finds the "pandaboard", applies
the flags.

You'll have to keep the JSON file pretty simple to be able to make
both CFLAGS and Internal Clang flags from the same source. Keeping
only standard names like "fpu": "vfpv3" is the best, then the script
and the Clang module know what to do by themselves.

hope this helps.

cheers,
--renato

Hi,

I started implementing the first step. I decided to implement a distinct tool(from what Clang does) that gathers system information. This is because I didn’t find the Clang code that does this(searched for a pointer in docs), also I think it is easier to write such a tool in Python instead of modifying the Clang code to output that info. Anyway, my decision is debatable, what do you think about it?

I will keep the source code in this repo: https://github.com/ner0x652/llvm-prohemium

Please, keep in mind that I am not a Python expert so, may be, the skeleton I wrote is not as good as you hope(want). Any support and advice is welcomed.

Thank you!

În Mar, 13 oct. 2015 la 18:29, Renato Golin <renato.golin@linaro.org> a scris:

I started implementing the first step. I decided to implement a distinct
tool(from what Clang does) that gathers system information. This is because
I didn't find the Clang code that does this(searched for a pointer in docs),

It's not in Clang, but LLVM. Check llvm-src/lib/Support/Host.cpp.

also I think it is easier to write such a tool in Python instead of
modifying the Clang code to output that info. Anyway, my decision is
debatable, what do you think about it?

Python is ok for prototypes. Once the logic is agreed between all
parties, we can move inside Clang, then it will have to be C++.

Please, keep in mind that I am not a Python expert so, may be, the skeleton
I wrote is not as good as you hope(want). Any support and advice is
welcomed.

Me neither. We'll rely on the community for cross-checking that. :slight_smile:

cheers,
--renato

Hi,

Oh, I will change the license.

Thank you for all your help!

Cheers,

Cornel.

În Vin, 16 oct. 2015 la 10:01, Renato Golin <renato.golin@linaro.org> a scris: