= Introduction = This page outlines the efforts taken to maintain OpenVPN's quality without excessive compromises on development speed. = Static testing = Static testing usually refers to [http://en.wikipedia.org/wiki/Static_code_analysis static code analysis], which is baked in into our [DeveloperDocumentation development process] in the form of mandatory ACK process every patch has to go through. The ACK process not only improves code quality, it also prevents highly specialized or rarely used features from polluting the codebase. In addition OpenVPN's codebase is scanned using [http://scan.coverity.com/ Coverity Scan] which detects many potential security vulnerabilities. = Dynamic testing = == Dedicated black-box tests == Dynamic black-box testing means trying out an application and verifying if it works as intended. In closed-source software development which is organized around a [http://en.wikipedia.org/wiki/Waterfall_model waterfall model] there are usually dedicated testers who do various scripted or intuitive tests to verify an application works as intended. This usually happens just before launch. In complex applications (such as OpenVPN) testing even a small fraction of functionality would be impractical and very costly. Fortunately, in [http://en.wikipedia.org/wiki/Lean_software_development Lean software development] methodologies such as [http://en.wikipedia.org/wiki/Scrum_%28development%29 Scrum] and especially in community-driven OSS development doing ''extensive'', dedicated testing is in general just a waste of time. It is replaced by * Constant quality assurance achieved with static whitebox technique (e.g. code reviews) * Testing in real environments (by users) This said, a minimal amount of dedicated, dynamic testing (a.k.a. [http://en.wikipedia.org/wiki/Smoke_testing smoke testing]) goes into each release to catch the most obvious errors. == Testing in real environments == In OpenVPN (and most other open source projects), the stability of stable releases is ensured with ''real-life testing'' by it's users during all phases of software development, starting from development code in Git and leading into stable releases. There are at least two kinds of ''barriers'' to using pre-release code: * '''Psychological barriers''' * Risk avoidance * '''Technical barriers''' * Unfamiliarity with required tools (e.g. Git) * Difficulty of deployment, e.g. building software from sources (especially on Windows) This means that the closer were getting closer to release, the more people we can expect to be testing the codebase. The figures below are not based on any real calculations and are only rough estimates: ||'''Git'''||'''Snapshots'''||'''Alpha'''||'''Beta'''||'''RC'''||'''Release'''|| ||0.1%||0.2%||1%||5%||10%||100%|| Use of snapshots help overcome technical barriers. As users are often reluctant to run development code that has to be compiled manually, "unstable" releases need to be pushed out as quickly as possible so that it gets into as wide circulation as possible and as fast as possible. This means bugs will be found and fixed quicker, so that new releases can be made quickly. This is illustrated below: The figures in the table are ''very rough'' approximations of usage == Continuous integration == The project also has a [http://trac.buildbot.net/ Buildbot] buildmaster, which drives several buildslaves. These together form a [http://en.wikipedia.org/wiki/Continuous_integration continuous integration] environment for OpenVPN. Each of these buildslaves is running a different operating system, and every commit to the OpenVPN Git repository triggers a build on each. After the build, each buildslave's newly compiled openvpn instance tries to connect to a test server using several different configurations. This ensures that * OpenVPN builds properly on a variety of platforms * Basic functionality is unaffected by commits == Unit testing == At the moment, there is no coherent set of [http://en.wikipedia.org/wiki/Unit_test unit tests] to spot regressions. One option would be to use [http://cunit.sourceforge.net/index.html CUnit] or similar unit test framework to cover the most commonly used and/or critical codepaths.