buildbot deployment: gsutil: Anonymous caller does not have storage.objects.create access to lldb_test_traces

Hello,

trying a test deployment of a buildbot but I do not see how to deploy the
final:
  upload test traces './uploadTestTrace.sh 43 ...' failed ( 10 secs )
  http://lab.llvm.org:8014/builders/lldb-x86_64-fedora-28-cmake/builds/43/steps/upload%20test%20traces/logs/stdio

I was following this guide but various steps are missing there, I will submit
them after I get it running.
  How To Add Your Build Configuration To LLVM Buildbot Infrastructure — LLVM 16.0.0git documentation

I have done for the last step:
  wget https://storage.googleapis.com/pub/gsutil.tar.gz
  ./setup.py install --user

But maybe it wants also:
  gsutil config
But I have no idea which Google Cloud Platform Project ID should I enter,
additionally it even wants from me to pay for Google Cloud Platform.

Thanks,
Jan Kratochvil

This step is very specific to our (android studio) buildbot setup, and
it uploads the test results to a google cloud account only accessible
to our team. If you wanted to do something like that, you'd have to
create your own account and set it up similarly.

However, I strongly suspect that you do not want this, and just want
to set up a buildbot that runs the lldb tests. In that case, this step
is completely unnecessary for you, and can be removed from the config
for your bot. The reason it hasn't been made optional in this slave
factory yet, is that you're the first person except us who wants to
set up a buildbot which runs tests (free/netbsd bots use the same
config too, but they don't run tests).

*However*, for setting up a new bot, I'd recommend not using this
particular slave factory (getLLDBScriptCommandsFactory) at all,
because it's heavily customized for our use case (*), and very
different from how typical llvm buildbots are set up. You might be
better off setting up a new factory, which just does the typical
checkout+build+(optional) test steps, and avoids all of this mess.
Maybe then even the *BSD bots will migrate to that (I tried to warn
them when they started running into the same issues as you are now,
but to no avail). Feel free to ping me on IRC if you want to know more
about this.

(*) some of this is complexity is warranted as the config also
supports running tests on a remote android, but overall, if I was
setting up a fresh bot now, I wouldn't have used this config either.

pl

However, I strongly suspect that you do not want this,

Right.

*However*, for setting up a new bot, I'd recommend not using this
particular slave factory (getLLDBScriptCommandsFactory) at all,
because it's heavily customized for our use case (*), and very
different from how typical llvm buildbots are set up. You might be
better off setting up a new factory, which just does the typical
checkout+build+(optional) test steps, and avoids all of this mess.

OK. For development of these new steps I guess I should run my own buildbot
master instance? As otherwise that will be probably several/many commits to
zorg repo (+requested buildbot master restarts) and I may screw up something
along.

Thanks,
Jan Kratochvil

Yes, that would definitely be the best, but last time I tried that, I
couldn't get my master instance to run, for any approximation of the
word "run" (which is part of the reason why I haven't done anything
about this slave factory, even though I really don't like it)..

Hello,

I need a testing local buildbot instance to develop a buildbot slave config:

> > *However*, for setting up a new bot, I'd recommend not using this
> > particular slave factory (getLLDBScriptCommandsFactory) at all,
> > because it's heavily customized for our use case (*), and very
> > different from how typical llvm buildbots are set up. You might be
> > better off setting up a new factory, which just does the typical
> > checkout+build+(optional) test steps, and avoids all of this mess.
>
> OK. For development of these new steps I guess I should run my own buildbot
> master instance? As otherwise that will be probably several/many commits to
> zorg repo (+requested buildbot master restarts) and I may screw up something
> along.

Yes, that would definitely be the best, but last time I tried that, I
couldn't get my master instance to run, for any approximation of the
word "run" (which is part of the reason why I haven't done anything
about this slave factory, even though I really don't like it)..

I have found buildbot versions different than 0.8.5 are incompatibile with
LLVM infrastructure/configs so to run 0.8.5 on Fedora 28 x86_64 I have
backported:
  https://people.redhat.com/jkratoch/buildbot-0.8.5-fix.patch
  https://people.redhat.com/jkratoch/buildbot-0.8.5-fix2.patch

So I downloaded zorg from LLVM and set it up
  [buildbot@host1 ~]$ ls -l lldbmaster
  lrwxrwxrwx 1 buildbot buildbot 32 Aug 14 18:55 lldbmaster -> zorg-git/buildbot/osuosl/master/
  [buildbot@host1 ~]$ ls -l lldbmaster/
  total 76
  -rw-r--r-- 1 buildbot buildbot 878 Aug 14 15:25 buildbot.tac
  drwxr-xr-x 2 buildbot buildbot 4096 Aug 14 19:01 config
  -rw-r--r-- 1 buildbot buildbot 9552 Aug 14 15:25 master.cfg
  drwxr-xr-x 2 buildbot buildbot 4096 Aug 14 15:25 public_html
  -rw-r--r-- 1 buildbot buildbot 465 Aug 14 15:25 README.txt
  drwxr-xr-x 2 buildbot buildbot 4096 Aug 14 15:25 templates
  -rw-r--r-- 1 buildbot buildbot 34088 Aug 14 19:01 twistd.log
  -rw------- 1 buildbot buildbot 7 Aug 14 19:01 twistd.pid
  lrwxrwxrwx 1 buildbot buildbot 28 Aug 14 19:00 zorg -> /home/buildbot/zorg-git/zorg
with zorg-git directory from https://llvm.org/git/zorg.git patched as attached
but then I still get:

1b (568 Bytes)

Hi Jan,

The public master uses zorg/buildbot. The config files are here:
https://llvm.org/svn/llvm-project/zorg/trunk/buildbot/osuosl/master/

It might be easier for you to use the silent(staging) master to test your slaves against. It is there for this exact purpose.

Here is info on how to point your salve to silent master.

http://llvm.org/docs/HowToAddABuilder.html

Thanks

Galina