Version 24 (modified by 10 years ago) (diff) | ,
---|
Table of Contents
Introduction
This page outlines the efforts taken to maintain OpenVPN's code quality without excessive compromises on development speed.
Static testing
Peer review
Static testing usually refers to static code analysis, which is baked in into our development process in the form of mandatory ACK process which every patch has to go through. The ACK process not only improves code quality, it also prevents highly specialized or rarely used features from polluting the codebase. Code reviews are especially important for patches coming from non-core contributors who may not be familiar with OpenVPN's coding practices.
Automated static testing
OpenVPN's codebase is scanned using Coverity Scan which detects many potential security vulnerabilities.
Dynamic testing
Dedicated black-box tests
Dynamic black-box testing means trying out an application and verifying if it works as intended. In closed-source software development which is organized around a waterfall model there are usually dedicated testers who do various scripted or intuitive/exploratory tests to verify that an application works as intended. This usually happens just before launch. In complex applications (such as OpenVPN) testing even a small fraction of functionality would be impractical and very costly. Fortunately, in Lean software development methodologies such as Scrum and especially in community-driven OSS development doing extensive dedicated testing is in general just a waste of time. It is replaced by
- Constant quality assurance achieved with static whitebox technique (e.g. code reviews)
- Testing in real environments (by users)
This said, a small amount of dedicated, dynamic testing (a.k.a. smoke testing) goes into each release to catch the most obvious errors:
Operating system | Architecture | Tester | List of tests | Notes |
Windows XP installer | i388 | mattock (IRC) | Uninstall/install, OpenVPN-GUI connect/disconnect, service connect/disconnect | |
Windows 7 installer | amd64 | mattock (IRC) | Same as for Windows XP, above | |
Ubuntu/Debian | i386/amd64 | mattock (IRC) | Install, upgrade, service connect/disconnect | 2-3 of the official packages are tested |
All tests are performed against an OpenVPN 2.1.3 server that authenticates from LDAP and uses tls-auth.
If you want to help with pre-release testing please don't hesitate to contact the developers about it.
Performance tests
In the past performance tests have been conducted to measure OpenVPN performance:
We don't currently have a reliable test network which we could use to detect small performance regressions. Please don't hesitate to contact us if you think you could help us out with this problem.
Testing in real environments
In OpenVPN (and most other open source projects), the stability of stable releases is ensured with real-life testing by it's users during all phases of software development, starting from patches sent to the mailing list, followed by development code in Git and leading into stable releases. There are at least two kinds of barriers to using pre-release code:
- Psychological barriers
- Risk avoidance
- Technical barriers
- Unfamiliarity with required tools (e.g. Git)
- Difficulty of deployment, e.g. building software from sources (especially on Windows)
This means that the closer were getting closer to release, the more people we can expect to be testing the codebase. The figures below are not based on any real data and can only be considered rough estimates:
Git | Snapshots | Alpha | Beta | RC | Release |
0.1% | 0.2% | 1% | 5% | 10% | 99% |
Use of snapshots help overcome some of the technical barriers. The only way to overcome psychological barriers is to speed up the release cycle. This results in new features get into wide circulation faster, which in turn results into issues being reported more quickly. This also gives more confidence in integrity of stable releases. On the flipside, more bugs will probably end up in the initial versions of the stable releases.
We do not currently provide development snapshots (binaries), although doing that would definitely be beneficial. If you're interested in building such a snapshots please let us know about it.
Continuous integration
The project also has a Buildbot buildmaster, which drives several buildslaves. These together form a continuous integration environment for OpenVPN. Each of these buildslaves is running a different operating system, and every commit to the OpenVPN Git repository triggers a build on each. In addition each openvpn binary built with default configure options makes connections to several test servers using several different configurations. This ensures that
- OpenVPN builds properly on a variety of platforms
- The very basic functionality is not horribly broken
Here's a full list of build tests currently (June 2014) in use:
- builder-arch-amd64-stable-master
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-arch-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-arch-amd64-stable-master--disable-lz4
- builder-arch-amd64-stable-master--disable-lzo
- builder-arch-amd64-stable-master--disable-lzo
- builder-arch-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-arch-amd64-stable-master--disable-lzo--disable-management
- builder-arch-amd64-stable-master--disable-management
- builder-arch-amd64-stable-master--disable-snappy
- builder-arch-amd64-stable-master--disable-ssl
- builder-arch-amd64-stable-master--disable-ssl--disable-lzo
- builder-arch-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-arch-amd64-stable-master--disable-ssl--disable-management
- builder-arch-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-arch-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-arch-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-arch-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-centos-6-amd64-stable-master
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-centos-6-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-centos-6-amd64-stable-master--disable-lz4
- builder-centos-6-amd64-stable-master--disable-lzo
- builder-centos-6-amd64-stable-master--disable-lzo
- builder-centos-6-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-centos-6-amd64-stable-master--disable-lzo--disable-management
- builder-centos-6-amd64-stable-master--disable-management
- builder-centos-6-amd64-stable-master--disable-snappy
- builder-centos-6-amd64-stable-master--disable-ssl
- builder-centos-6-amd64-stable-master--disable-ssl--disable-lzo
- builder-centos-6-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-centos-6-amd64-stable-master--disable-ssl--disable-management
- builder-centos-6-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-centos-6-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-centos-6-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-centos-6-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-cron2-freebsd-74-amd64-stable-master
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-lz4
- builder-cron2-freebsd-74-amd64-stable-master--disable-lzo
- builder-cron2-freebsd-74-amd64-stable-master--disable-lzo
- builder-cron2-freebsd-74-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-cron2-freebsd-74-amd64-stable-master--disable-lzo--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-snappy
- builder-cron2-freebsd-74-amd64-stable-master--disable-ssl
- builder-cron2-freebsd-74-amd64-stable-master--disable-ssl--disable-lzo
- builder-cron2-freebsd-74-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--disable-ssl--disable-management
- builder-cron2-freebsd-74-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-cron2-freebsd-74-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-cron2-freebsd-74-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-cron2-freebsd-74-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-cron2-freebsd-82-amd64-stable-master
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-lz4
- builder-cron2-freebsd-82-amd64-stable-master--disable-lzo
- builder-cron2-freebsd-82-amd64-stable-master--disable-lzo
- builder-cron2-freebsd-82-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-cron2-freebsd-82-amd64-stable-master--disable-lzo--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-snappy
- builder-cron2-freebsd-82-amd64-stable-master--disable-ssl
- builder-cron2-freebsd-82-amd64-stable-master--disable-ssl--disable-lzo
- builder-cron2-freebsd-82-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--disable-ssl--disable-management
- builder-cron2-freebsd-82-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-cron2-freebsd-82-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-cron2-freebsd-82-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-cron2-freebsd-82-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-cron2-freebsd-90-amd64-stable-master
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-lz4
- builder-cron2-freebsd-90-amd64-stable-master--disable-lzo
- builder-cron2-freebsd-90-amd64-stable-master--disable-lzo
- builder-cron2-freebsd-90-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-cron2-freebsd-90-amd64-stable-master--disable-lzo--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-snappy
- builder-cron2-freebsd-90-amd64-stable-master--disable-ssl
- builder-cron2-freebsd-90-amd64-stable-master--disable-ssl--disable-lzo
- builder-cron2-freebsd-90-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--disable-ssl--disable-management
- builder-cron2-freebsd-90-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-cron2-freebsd-90-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-cron2-freebsd-90-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-cron2-freebsd-90-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-cron2-netbsd-51-amd64-stable-master
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-lz4
- builder-cron2-netbsd-51-amd64-stable-master--disable-lzo
- builder-cron2-netbsd-51-amd64-stable-master--disable-lzo
- builder-cron2-netbsd-51-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-cron2-netbsd-51-amd64-stable-master--disable-lzo--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-snappy
- builder-cron2-netbsd-51-amd64-stable-master--disable-ssl
- builder-cron2-netbsd-51-amd64-stable-master--disable-ssl--disable-lzo
- builder-cron2-netbsd-51-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--disable-ssl--disable-management
- builder-cron2-netbsd-51-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-cron2-netbsd-51-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-cron2-netbsd-51-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-cron2-netbsd-51-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-lz4 --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-lzo --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-lzo--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-lzo--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-snappy --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-ssl--disable-lzo--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-ssl--disable-lzo--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-ssl--disable-management--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--disable-ssl--disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl --disable-plugin-auth-pam
- builder-cron2-openbsd-49-i386-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl --disable-plugin-auth-pam
- builder-cron2-opensolaris-10-i386-stable-master
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-lz4
- builder-cron2-opensolaris-10-i386-stable-master--disable-lzo
- builder-cron2-opensolaris-10-i386-stable-master--disable-lzo
- builder-cron2-opensolaris-10-i386-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-cron2-opensolaris-10-i386-stable-master--disable-lzo--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-snappy
- builder-cron2-opensolaris-10-i386-stable-master--disable-ssl
- builder-cron2-opensolaris-10-i386-stable-master--disable-ssl--disable-lzo
- builder-cron2-opensolaris-10-i386-stable-master--disable-ssl--disable-lzo--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--disable-ssl--disable-management
- builder-cron2-opensolaris-10-i386-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-cron2-opensolaris-10-i386-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-cron2-opensolaris-10-i386-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-cron2-opensolaris-10-i386-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-debian-6-i386-stable-master
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-management
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-debian-6-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-debian-6-i386-stable-master--disable-lz4
- builder-debian-6-i386-stable-master--disable-lzo
- builder-debian-6-i386-stable-master--disable-lzo
- builder-debian-6-i386-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-debian-6-i386-stable-master--disable-lzo--disable-management
- builder-debian-6-i386-stable-master--disable-management
- builder-debian-6-i386-stable-master--disable-snappy
- builder-debian-6-i386-stable-master--disable-ssl
- builder-debian-6-i386-stable-master--disable-ssl--disable-lzo
- builder-debian-6-i386-stable-master--disable-ssl--disable-lzo--disable-management
- builder-debian-6-i386-stable-master--disable-ssl--disable-management
- builder-debian-6-i386-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-debian-6-i386-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-debian-6-i386-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-debian-6-i386-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-debian-7-i386-stable-master
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-management
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-debian-7-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-debian-7-i386-stable-master--disable-lz4
- builder-debian-7-i386-stable-master--disable-lzo
- builder-debian-7-i386-stable-master--disable-lzo
- builder-debian-7-i386-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-debian-7-i386-stable-master--disable-lzo--disable-management
- builder-debian-7-i386-stable-master--disable-management
- builder-debian-7-i386-stable-master--disable-snappy
- builder-debian-7-i386-stable-master--disable-ssl
- builder-debian-7-i386-stable-master--disable-ssl--disable-lzo
- builder-debian-7-i386-stable-master--disable-ssl--disable-lzo--disable-management
- builder-debian-7-i386-stable-master--disable-ssl--disable-management
- builder-debian-7-i386-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-debian-7-i386-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-debian-7-i386-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-debian-7-i386-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-fedora-19-i386-stable-master
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-management
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-fedora-19-i386-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-fedora-19-i386-stable-master--disable-lz4
- builder-fedora-19-i386-stable-master--disable-lzo
- builder-fedora-19-i386-stable-master--disable-lzo
- builder-fedora-19-i386-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-fedora-19-i386-stable-master--disable-lzo--disable-management
- builder-fedora-19-i386-stable-master--disable-management
- builder-fedora-19-i386-stable-master--disable-snappy
- builder-fedora-19-i386-stable-master--disable-ssl
- builder-fedora-19-i386-stable-master--disable-ssl--disable-lzo
- builder-fedora-19-i386-stable-master--disable-ssl--disable-lzo--disable-management
- builder-fedora-19-i386-stable-master--disable-ssl--disable-management
- builder-fedora-19-i386-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-fedora-19-i386-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-fedora-19-i386-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-fedora-19-i386-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-ubuntu-1004-amd64-stable-master
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-lz4
- builder-ubuntu-1004-amd64-stable-master--disable-lzo
- builder-ubuntu-1004-amd64-stable-master--disable-lzo
- builder-ubuntu-1004-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-ubuntu-1004-amd64-stable-master--disable-lzo--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-snappy
- builder-ubuntu-1004-amd64-stable-master--disable-ssl
- builder-ubuntu-1004-amd64-stable-master--disable-ssl--disable-lzo
- builder-ubuntu-1004-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1004-amd64-stable-master--disable-ssl--disable-management
- builder-ubuntu-1004-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-ubuntu-1004-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-ubuntu-1004-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-ubuntu-1004-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-ubuntu-1204-amd64-stable-master
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-lz4
- builder-ubuntu-1204-amd64-stable-master--disable-lzo
- builder-ubuntu-1204-amd64-stable-master--disable-lzo
- builder-ubuntu-1204-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-ubuntu-1204-amd64-stable-master--disable-lzo--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-snappy
- builder-ubuntu-1204-amd64-stable-master--disable-ssl
- builder-ubuntu-1204-amd64-stable-master--disable-ssl--disable-lzo
- builder-ubuntu-1204-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1204-amd64-stable-master--disable-ssl--disable-management
- builder-ubuntu-1204-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-ubuntu-1204-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-ubuntu-1204-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-ubuntu-1204-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
- builder-ubuntu-1404-amd64-stable-master
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-crypto--disable-ssl--disable-ssl--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-lz4
- builder-ubuntu-1404-amd64-stable-master--disable-lzo
- builder-ubuntu-1404-amd64-stable-master--disable-lzo
- builder-ubuntu-1404-amd64-stable-master--disable-lzo --disable-snappy --disable-lz4 --enable-comp-stub
- builder-ubuntu-1404-amd64-stable-master--disable-lzo--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-snappy
- builder-ubuntu-1404-amd64-stable-master--disable-ssl
- builder-ubuntu-1404-amd64-stable-master--disable-ssl--disable-lzo
- builder-ubuntu-1404-amd64-stable-master--disable-ssl--disable-lzo--disable-management
- builder-ubuntu-1404-amd64-stable-master--disable-ssl--disable-management
- builder-ubuntu-1404-amd64-stable-master--with-crypto-library=openssl --enable-crypto --disable-ssl
- builder-ubuntu-1404-amd64-stable-master--with-crypto-library=openssl --enable-crypto --enable-ssl
- builder-ubuntu-1404-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --disable-ssl
- builder-ubuntu-1404-amd64-stable-master--with-crypto-library=polarssl --enable-crypto --enable-ssl
At the moment the most common operating system variants are quite thoroughly covered. If you have access to really funky hardware and software combinations please don't hesitate to contact us and ask if adding another buildslave would make sense.
In addition each commit triggers a Linux -> Windows cross-compile test using the generic buildsystem. This helps prevent the Windows builds from being horribly broken.
Unit testing
At the moment, there is no coherent set of unit tests to spot regressions. One option would be to use CUnit or similar unit test framework to cover the most commonly used and/or critical codepaths.