FreeBSD LLDB buildbot

Years ago there was a FreeBSD LLDB buildbot managed by Galina; it was
decommissioned quite some time ago. I've been running one since then,
but it was never set up to run tests, only build.

Over the weekend I tried to get it to run tests, but ended up with a
broken installation. I've started over in a clean FreeBSD jail and
have it configured again, but it appears that FreeBSD now has a newer
version of the buildbot worker package (0.9.11) in the package
collection that is not compatible with the server; the Python 3
version reported:

remoteFailed: [Failure instance: Traceback from remote host --
Traceback unavailable
buildbot_worker.base.UnknownCommand: unrecognized WorkerCommand 'b'shell''
]

I installed the Python 2 buildbot-worker 0.9.11 and it reports:

exceptions.AssertionError: Unexpected usePTY argument value:
'slave-config'. Expected boolean.
exceptions.AssertionError: Unexpected usePTY argument value:
'slave-config'. Expected boolean.

It seems there's a number of interop problems between different
Buildbot versions and Python 2 / Python 3. What is the best way to
install a compatible buildbot worker?

Hi Ed,

Please try to install pip to get buildslave v0.8.12.
Something like this should work:
pip install ‘buildbot-slave <= 0.8.12’

Thanks

Galina

Thanks Galina. I now have a jail running FreeBSD 11.1-Release hosting
the original lldb-amd64-ninja-freebsd11 configuration, so we're at
least back to a green build-only FreeBSD buildbot.
http://lab.llvm.org:8011/builders/lldb-amd64-ninja-freebsd11

For reference (most likely for my future self, when I have to set up
another instance), this is how I set it up:

1. Create a FreeBSD 11.1-Release jail as described in
https://www.freebsd.org/doc/handbook/jails-build.html. (As an aside, I
found some outdated information in the FreeBSD handbook jails section;
changes are in progress but noted in case anyone else wants to try.)

2. Install required packages:
# pkg install cmake ninja swig30 subversion py27-pip

3. installed buildbot-slave via pip, as described above. This jail is
used exclusively for this buildbot instance so there is no concern
installing to the system-wide location.

4. Recreate buildbot.tac
$ buildslave create-slave /usr/home/buildbot/scratch/
lab.llvm.org:9990 lldb-amd64-ninja-freebsd11 <password>

5. I copied the scripts (updateScripts.sh, checkoutSource.sh, etc.)
from the previous buildbot installation (into scratch/scripts/). I
think we should host all scripts for the builders in either the lldb
or zorg repo - is there a reason they're not?

6. Add updateScripts.sh to path (I symlinked it into ~/bin).

7. Start the buildbot worker process
$ buildslave start

The host is running a FreeBSD 12-CURRENT kernel and to enable tests I
expect I'll create a new FreeBSD 12 builder, which can be initially
connected to the staging master.

2. Install required packages:
# pkg install cmake ninja swig30 subversion py27-pip

Also gmake and bash.

The host is running a FreeBSD 12-CURRENT kernel and to enable tests I
expect I'll create a new FreeBSD 12 builder, which can be initially
connected to the staging master.

I've sent connection details for lldb-amd64-ninja-freebsd-wip-12, and
will post a patch for slaves.py and builders.py soon.

There is one problem reported by the lit tests with my jail approach:

[ RUN ] SocketTest.TCPGetAddress
/usr/home/buildbot/scratch/scratch/llvm/tools/lldb/unittests/Host/SocketTest.cpp:207:
Failure
      Expected: "127.0.0.1"
To be equal to: socket_a_up->GetRemoteIPAddress().c_str()
      Which is: "192.168.11.4"

In a (default configuration) jail FreeBSD maps 127.0.0.1 to the jail's
IP address. Support for independent virtualized network stacks is a
more recent addition to FreeBSD and it looks like I'll need to set
that up here (or use full virtual machines).

The existence of these scripts is a fairly non-standard feature in the
llvm buildbot world. They were created to provide a layer of
indirection between the buildmaster and the slave. The main reason for
that was that there's a fairly elaborate dance you need to do to setup
remote testing. Implementing this as a series of buildbot steps was
tricky and required a master restart everytime you needed to change
things. I wasn't involved in setting this up back then, but I guess at
some point someone just got fed up with that and implemented this.

I'm don't think we should be recommending using these for new
buildbots. There is a more standard lldb buildslave factory class in
the zorg repo that does not need these scripts (but probably needs
some resuscitation) and that's what I would recommend for the regular
setup. The scripted factory can stay for the more advanced uses, but
here I would say it's up to the buildslave maintainer to create these
scripts for himself.