Contributing a buildbot for the BPF backend

Hi everyone,
I am working with Alexei Starovoitov to contribute an LLVM buildbot
for the experimental BPF backend.
I am following the steps at [1] to setup a buildbot and I was mostly
successful: I was able to setup a slave and a temporary master to
check its base config.

Now I'm not sure about the next step: patching the "slaves.py" and
"builders.py" files in zorg.

* Is there a guide I can follow to understand the format of these
files? For example, being BPF an experimental backend, I need to
provide extra parameters to cmake at build time.

* Also, is there a way to test these changes in zorg in a local
environment before submitting the patch for review?

Thank you!

[1] http://llvm.org/docs/HowToAddABuilder.html

Regards,

Marco Leogrande

inaccuracies, logical fallacies and repetitions.

Hi all,
I was trying to make some progress on this front and I spent some more
time on it.

* I think I managed to configure the buildslave correctly; any chance
there is a way to test that before the buildmaster is configured?

* I was trying to figure out where/how to add the configuration in
zorg. slaves.py looks reasonably easy to configure, but builders.py
looks more convoluted. I think the attached patch is a start for the
build part, but of course I have no idea if it makes any sense :slight_smile:
The idea would be to replicate the following workflow (build
experimental backend + tests):

cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/
make -jX
./bin/llvm-lit test/CodeGen/BPF/

Any suggestion would be appreciated.

Thanks!

Regards,

Marco Leogrande

inaccuracies, logical fallacies and repetitions.

01-zorg-bpf.patch (1.73 KB)

Hello Marco,

Welcome aboard!

slaves.py keeps the build slaves definitions, builders.py keeps the builders definitions. You have to have both.

The steps of adding a new slave is here: http://llvm.org/docs/HowToAddABuilder.html. Please make sure you done the step # 10 before bringing your slave up, otherwise it wouldn’t be authorized by the master and will be blacklisted after multiple failures.

To make sure the slave is ready, you can start from manually building and testing everything you want the build to do on that slave in context of the account the build slave will be running in.
If you succeed, then the slave is likely ready.

You patch doesn’t look right.
getLLVMBuildFactory does not support cmake, so it wouldn’t do what you think it will. For the same reason your slave properties (jobs and loadaverage) wouldn’t work.

See zorg/buildbot/builders/LLVMBuilder.py for more details.

You either build LLVM with configure (this is what the existing LLVM factory does), add a new CMakeBuildFactory to support cmake, or build LLVM+clang with cmake by using a clang build factory.

If you use any of the existing build factories, you do not really need a local master. You can add your builder and your slave to the lab.llvm.org master and move from there.
As the first step you will be adding your builder as experimental anyway, so you will have a chance to work out possible issues and instabilities before promoting it to a stable one.

You do need a local master to be able to debug and test if you develop zorg, i.e. write a new CMakeBuildFactory and such, though.

Hope this helps.

Thanks

Galina

Hi Galina,

thanks for the info!

I think we will go for the "LLVM+clang with cmake" approach, since the
BPF target has been marked non-experimental in the meantime, and we
are interested in building clang as well.
I will provide the info requested in step #10 separately and experiment a bit.

Thanks.

Regards,

Marco Leogrande
(mobile) +1(408)475-1341

inaccuracies, logical fallacies and repetitions.

Galina,
thanks again for your help.

I have cooked a new patch that should make a bit more sense.
I have added the new builder under
_get_experimental_scheduled_builders(), as the comment above that
function suggested me it would be a good idea. :slight_smile:
Does it look a reasonable start?

Regards,

Marco Leogrande

inaccuracies, logical fallacies and repetitions.

0001-Add-builder-and-slave-for-experimental-BPF-builder.patch (1.97 KB)

LGTM.

Please commit.

Thanks

Galina

Thank you.
Alexei just committed the change to SVN.

Regards,

Marco Leogrande

inaccuracies, logical fallacies and repetitions.

Hi Marco,

Your slave should be able to attach to llvm master now.

Please let me know if there is any problems with this.

Thanks

Galina

Hi Galina,
I can see the buildslave in the waterfall display, and one build was
already issued and completed successfully.

I will give it a deeper look at the output and monitor in the next few days.

Thanks!

Regards,

Marco Leogrande

inaccuracies, logical fallacies and repetitions.