Version 4 (modified by Samuli Seppänen, 11 years ago) (diff)



This page outlines the efforts taken to maintain OpenVPN's quality without excessive compromises on development speed.

Static testing

Static testing usually refers to static code analysis, which is baked in into our development process in the form of mandatory ACK process every patch has to go through. In addition, OpenVPN codebase is scanned using Coverity Scan which can detect many potential security vulnerabilities.

Dynamic testing

Dynamic (black-box) testing means trying out an application and verifying if it works as intended. In closed-source software development which is organized around a waterfall model there are usually dedicated testers who do various scripted or intuitive tests to verify an application works as intended. This usually happens just before launch. In complex applications (such as OpenVPN) testing even a small fraction of functionality would be impractical and very costly. Fortunately, in Lean software development methodologies such as Scrum and community-driven OSS development doing extensive, dedicated testing is in general just a waste of time. That said, a minimal amount of dedicated testing (a.k.a. smoke testing) goes into each release to make sure they work as intended.

In OpenVPN (and most other open source projects), the stability of stable releases (e.g. 2.1, 2.2) is ensured with real-life testing by it's users during all phases of software development, starting from development code in Git and leading into stable releases. Even though only a small subset of users will be running the development, alpha, beta or rc code, they will still be able to catch the most common issues. As users tend not to run development code that has to be compiled manually, "unstable" releases are pushed out as quickly as possible so that it gets into as wide circulation as possible as fast as possible. This means bugs will be found and fixed quicker, so that new releases can be made quickly.

The project also has a Buildbot buildmaster, which drives several buildslaves. Each of these buildslaves is running a different operating system, and every commit to the OpenVPN Git repository triggers a build on each. After the build, each buildslave's openvpn tries to connect to a test server using several different configurations, thus ensuring that basic functionality is unaffected by the commit.