Help needed setting up a new buildbot for linux-sparcv9 and other targets

Project Infrastructure LLVM Dev List Archives
1

Hello!

I would like to set up some additional buildbots for LLVM and Clang that are currently
missing in the current list builders [1], namely linux-sparcv9, linux-ppc32 and linux-mips
(and later linux-m68k).

I'm currently following the guide in [2] and I have installed the build dependencies for
LLVM as well as the buildbot package on a Debian sparc64 unstable machine (Sun T5220).

I'm currently stuck at this step:

$ buildbot-worker create-worker <buildbot-worker-root-directory> \
             lab.llvm.org:9990 \
             <buildbot-worker-access-name> \
             <buildbot-worker-access-password>

I'm wondering what to use as the "buildbot-worker-access-name" username.

And what builddir does the sentence

  "Please make sure your builder name and its builddir are unique through the file."

refer to? Isn't the build dir local to the builder machine only?

Thanks,
Adrian

2

Hey Galina - perhaps you could help John here?

3

Hello John,

I would like to set up some additional buildbots for LLVM and Clang that are currently
missing in the current list builders [1], namely linux-sparcv9, linux-ppc32 and linux-mips
(and later linux-m68k).

Thanks for working on this!

I’m wondering what to use as the “buildbot-worker-access-name” username.

Any distinct unused yet name which represents the build worker you are about to introduce is fine. For example, “linux-sparcv9”, “linux-ppc32”, “linux-mips”, and “linux-m68k” look good. You could specify builddir to be the same as the worker name. This would keep things simple.

I can assign passwords to your workers or you can choose whatever password you want and send directly to me. Please let me know if you need my public pgp key for secure communication.

We do not accept changes to zorg at this time because of the upgrade to a newer version of buildbot. The repository is locked and I’m in the middle of preparing a set of patches you will need to base your changes on. I hope this will be done early next week.

Anyway, once the llvm-zorg has all the changes for the new buildbot, you will need to propose a patch. Please note that the below explanation is based on the new file names which are not available yet in the github.

To add your workers to LLVM buildbot, you will need to propose a patch for buildbot/osuosl/master/config/workers.py which adds the workers, and for buildbot/osuosl/master/config/builders.py to add your build configurations which use your workers. If you are up to building self hosting clang and llvm, then please use UnifiedTreeBuilder.

If you will elaborate what build configuration you are after for your bots, I might give you a good example to follow or will help you with the patch.

I assume you already made sure you can build locally on each of those workers what you are after.

Thanks

Galina

4

Hi Galina!

Any distinct unused yet name which represents the build worker you are
about to introduce is fine. For example, "linux-sparcv9", "linux-ppc32",
"linux-mips", and "linux-m68k" look good. You could specify builddir to be
the same as the worker name. This would keep things simple.

Is there any naming scheme that the other buildbot-workers are using or are the
names just random?

And why does the buildddir have to be unique? Isn't that local on the buildbot-
worker?

I can assign passwords to your workers or you can choose whatever password
you want and send directly to me. Please let me know if you need my public
pgp key for secure communication.

Yes, your GPG key would be appreciated although I guess I can find it on the
public servers assigned to your email?

We do not accept changes to zorg at this time because of the upgrade to a
newer version of buildbot. The repository is locked and I'm in the middle
of preparing a set of patches you will need to base your changes on. I hope
this will be done early next week.

OK, noted.

Anyway, once the llvm-zorg has all the changes for the new buildbot, you
will need to propose a patch. Please note that the below explanation is
based on the new file names which are not available yet in the github.

To add your workers to LLVM buildbot, you will need to propose a patch for
buildbot/osuosl/master/config/workers.py which adds the workers, and for
buildbot/osuosl/master/config/builders.py to add your build configurations
which use your workers. If you are up to building self hosting clang and
llvm, then please use UnifiedTreeBuilder.

OK.

If you will elaborate what build configuration you are after for your bots,
I might give you a good example to follow or will help you with the patch.

Selfhosting LLVM and clang would be my goal on 64-bit SPARC on Linux.

The machine has 64 vCPUs and 32 GiB of RAM. Running Debian unstable with
the latest versions of GCC, CMake and Binutils. Ninja is available, too.

I assume you already made sure you can build locally on each of those
workers what you are after.

Yep. That works fine. Although there is currently one regression on SPARC
which I already reported to the list.

Adrian

5

Hi John,

Is there any naming scheme that the other buildbot-workers are using or are the names just random?

There is no enforced naming scheme. This is more up to the bot owner. If you want, you could use - or vice versa.

And why does the buildddir have to be unique? Isn’t that local on the buildbot-worker?

Here is the quote from the buildbot documentation:

`builddir`
Specifies the name of a subdirectory of the master’s basedir in which everything related to this builder will be stored. This holds build status information. If not set, this parameter defaults to the builder name, with some characters escaped. Each builder must have a unique build directory.
`workerbuilddir`
Specifies the name of a subdirectory (under the worker’s configured base directory) in which everything related to this builder will be placed on the worker. This is where checkouts, compiles, and tests are run. If not set, defaults to `builddir`. If a worker is connected to multiple builders that share the same `workerbuilddir`, make sure the worker is set to run one build at a time or ensure this is fine to run multiple builds from the same directory simultaneously.

Thanks

Galina

6

Hi Galina!

Is there any naming scheme that the other buildbot-workers are using or

are the names just random?

There is no enforced naming scheme. This is more up to the bot owner. If
you want, you could use <machine>-<operatingsystem> or vice versa.

OK. I was just wondering what the other machines are using so the names
would be consistent.

And why does the buildddir have to be unique? Isn't that local on the

buildbot-worker?

Here is the quote from the buildbot documentation:
builddirSpecifies the name of a subdirectory of the master’s basedir in
which everything related to this builder will be stored. This holds build
status information. If not set, this parameter defaults to the builder
name, with some characters escaped. Each builder must have a unique build
directory.workerbuilddirSpecifies the name of a subdirectory (under the
worker’s configured base directory) in which everything related to this
builder will be placed on the worker. This is where checkouts, compiles,
and tests are run. If not set, defaults to builddir. If a worker is
connected to multiple builders that share the same workerbuilddir, make
sure the worker is set to run one build at a time or ensure this is fine to
run multiple builds from the same directory simultaneously.

Alright, thanks. I'm already seeing what the other bots are using, so I'll stick
to that scheme [1].

I will still need your GPG key which is unfortunately not available on the public
servers.

Thanks,
Adrian