Buildbot log retention.

Hi all,

Can we increase the number of logs retained by the lldb buildbots? It’s currently holding on to ~200 builds per configuration. I’d like to see that increase to 1000 or even 2000.

Thoughts?

Thanks,

Vince

Hi Vince,

Unfortunately, this is not as easy as we’d want.

We are using Mac-mini as the buildmaster, and even if there is enough room on the disk, it might be not enough performance to deal with 5-10 times more data.

I’ll look onto this.

Most of the logs won’t ever be accessed, they’ll just be sitting somewhere on the filesystem. As long as you have plenty of disk space, I don’t anticipate any problems.

Even if it’s an issue to change this for everything, what about allowing it for specific builders via a configuration setting override?

The debian builder only has logs back ~29 hours ago, build 4197. The latest build is 4249. We’re retaining less than 60 builds, is that expected?

Hello Vince,

Unfortunately, that’s not that easy. Buildbot keeps all the builds statuses and all the build steps statuses along with the actual build logs. This information gets “pickled” - i.e. serialized to the file system and then deserialized back when consumed, for instance by the web UI, with some caching on top. Everything gets loaded before any filter could be applied.

I have changed the settings to keep the logs for all the statuses we keep (currently 200). This is safe and relatively “cheap” change at the “cost” of some room on a hard drive.

I will also play with the history horizon to see how the response time would change depending on various settings, It might be some sweet spot there we can get to.

what about allowing it for specific builders via a configuration setting override?

Zachary, please feel free to propose the patch. So far we have just few LLDB builders, and selectively increasing the horizon just for them, should not hurt much.

Other possible move would be upgrading the build master hardware for a faster disk subsystem.

Thanks

Galina