wiki:SettingUpBuildslave

Version 35 (modified by Samuli Seppänen, 6 years ago) (diff)

--

Introduction

OpenVPN project uses Buildbot (current version 0.8.5) to help increase code quality. Buildbot is a Python application that can work in either master or slave mode. The buildmaster is the core server which accepts connections from buildslaves and tells them what they should do. Typically the clients fetch latest sources and report any build problems to buildbot which in turn informs developers via email. In software engineering this is called Continous integration and helps prevent build problems go unnoticed for extended time periods. The clients (buildslaves) can and should run on a variety of hardware / OS platforms. For the server (buildmaster) the OS choice is largely irrelevant.

As the number of buildslaves can easily get out of hand, the OpenVPN project can make use of your help - see below for details.

Buildbot is described in more detail in the Buildbot manual.

Setting up the VPN connection

Our buildbot (buildmaster) is accessible only via OpenVPN. To connect your buildslave to it you need:

  1. Active OpenVPN community services user account
  2. A properly configured OpenVPN client
  3. Certificate authority (CA) key for the VPN
  4. TLS-auth (TA) key for the VPN
  5. Membership in the appropriate LDAP group

You can do 1. yourself. The rest will be taken care of for the most part by Samuli, our community manager at openvpn.net. He can also be reached at #openvpn and #openvpn-devel channels as "mattock".

Setting up a buildslave

Installing buildbot

Installing buildbot from distro repositories

Many Linux distributions have buildbot in their package repositories. Using the distro version is probably the easiest approach. Buildbot versions 0.7.x and 0.8.x are known to work with our current (0.8.5) buildmaster.

Installing buildbot using easy_install

A fairly easy and somewhat OS-agnostic way to install buildbot is to use easy_install: it should be available your OS'es software repository. Using easy_install will keep buildslave isolated from your OS'es package management and take care of dependencies for you. It is easiest (but not necessary) to run easy_install as root:

First switch to root account, e.g. using sudo or su:

$ sudo -s

Next test that easy_install should work as expected:

$ easy_install -n buildbot==0.8.5

If all went well, install buildbot for real:

$ easy_install buildbot==0.8.5

Once buildslave is installed you can and should switch to an ordinary user account.

Configuring the buildslave

Buildslave configuration is covered thoroughly in the the buildbot manual. Basically you just step through the "Creating a buildslave" section in the buildbot manual except that you should skip step 1 ("Set up the account") because the connectivity tests will require root access anyways.

Configuring the buildslave for connectivity tests

OpenVPN project's buildslaves run connectivity tests against several OpenVPN test servers once per on every commit. For these tests to work you need to run buildbot as root.

As the tests connect to remote OpenVPN servers you will need test certificates and a t_client.rc config file from the buildmaster admins. Once you've have the files, put them to /home/buildbot:

$ tree /home/buildbot
/home/buildbot
├── t_client.rc
├── test-ca.crt
├── test-client.crt
├── test-client.key
└── test-ta.key

0 directories, 5 files

Now make sure that the files

  • are named exactly as shown above or buildbot won't find them and will fail
  • are readable by root
  • have strict enough permissions to keep OpenVPN happy
    • 600 for test-client.key
    • 644 for other files

Now the admins will force a build and the associated connectivity tests on your buildslave. The first build is expected to fail, because the t_client.rc you were given first is a generic one. Once the IPv4 and IPv6 addresses the connectivity test servers give your client are known, the admins can send you a new t_client.rc tailored for you buildslave.

List of existing buildslaves

Here's a comprehensive list of Buildslaves already running (as of Apr 2014). There can be several buildslaves that have the same OS/architecture combination, but this is seldom necessary.

Operating systemVersionArchitecture24/7Connection testsProvided byNotes
Arch-amd64-Yesroentgen
Debian6 (squeeze)i386NoYesmattock
Debian7 (wheezy)i386NoYesmattock
Ubuntu10.04amd64NoYesmattock
Ubuntu12.04amd64NoYesmattock
CentOS6.xamd64NoYesmattock
FreeBSD7.4amd64YesYescron2ongoing
FreeBSD8.2amd64YesYescron2
FreeBSD9.0amd64YesYescron2
OpenBSD4.9i386YesYescron2
NetBSD5.1amd64YesYescron2

List of build permutations

There are too many to list. However, the builds that use default build flags also trigger connectivity tests.

Troubleshooting

In case your build fails, try running the same build steps manually to see what the problem is.

Source checkout

If you're running into odd problems with buildslave Git checkouts you can try manually running the same commands the buildslave would run. The exact steps can be found from the buildslave's Git logs (in the buildmaster web interface), but they should be similar to these:

$ mkdir /tmp/openvpntest
$ cd /tmp/openvpntest
$ git init
$ git fetch -t git://openvpn.git.sourceforge.net/gitroot/openvpn/openvpn.git +master
$ git reset --hard FETCH_HEAD
$ git branch -M master

If you're unable to run these commands manually then the buildslave won't be able to do it either.