Changes between Version 13 and Version 14 of OpenVPN_QA

Timestamp:

09/26/11 08:53:12 (13 years ago)

Author:

Samuli Seppänen

Comment:

--

OpenVPN_QA

@@ +1 @@
+[[TOC(inline, depth=1)]]
+
 = Introduction =
 
 This page outlines the efforts taken to maintain OpenVPN's quality without excessive compromises on development speed.
 
+The next major release (as of Sep 2011), OpenVPN 2.3, will contain significant low-level changes to OpenVPN 2.2.x and 2.1.x. Therefore lots of work is needed on the QA front to avoid regressions. The plan is to setup good QA environment for 2.3a -> 2.3GA development phase. This environment will contain several different parts:
+
 = Static testing =
+
+== Peer review ==
 
 Static testing usually refers to [http://en.wikipedia.org/wiki/Static_code_analysis static code analysis], which is baked in into our [DeveloperDocumentation development process] in the form of mandatory ACK process every patch has to go through. The ACK process not only improves code quality, it also prevents highly specialized or rarely used features from polluting the codebase.
 
-In addition OpenVPN's codebase is scanned using [http://scan.coverity.com/ Coverity Scan] which detects many potential security vulnerabilities.
+== Automated static testing ==
+
+OpenVPN's codebase is scanned using [http://scan.coverity.com/ Coverity Scan] which detects many potential security vulnerabilities.
 
 = Dynamic testing =
@@ -20 +28 @@
 This said, a minimal amount of dedicated, dynamic testing (a.k.a. [http://en.wikipedia.org/wiki/Smoke_testing smoke testing]) goes into each release to catch the most obvious errors.
 
+== Performance tests ==
+
+To ensure good performance, performance tests are in place. For details, look at the [wiki:PerformanceTesting performance testing wiki page].
+
 == Testing in real environments ==
 
@@ -30 +42 @@
   * Difficulty of deployment, e.g. building software from sources (especially on Windows)
 
-This means that the closer were getting closer to release, the more people we can expect to be testing the codebase. The figures below are not based on any real calculations and can only be considered rough estimates:
+This means that the closer were getting closer to release, the more people we can expect to be testing the codebase. The figures below are not based on any real data and can only be considered rough estimates:
 
 ||'''Git'''||'''Snapshots'''||'''Alpha'''||'''Beta'''||'''RC'''||'''Release'''||
 ||0.1%||0.2%||1%||5%||10%||99%||
 
-Use of snapshots help overcome some of the technical barriers. The only way to overcome psychological barriers is to speed up the release cycle. This results in new features get into wide circulation faster, which in turn results into issues being reported more quickly. This also gives more confidence in integrity of stable releases. 
+Use of snapshots help overcome some of the technical barriers. The only way to overcome psychological barriers is to speed up the release cycle. This results in new features get into wide circulation faster, which in turn results into issues being reported more quickly. This also gives more confidence in integrity of stable releases. On the flipside, more bugs will probably end up in the initial versions of the stable releases.
+
+In practice, the goal is to make live testing by users as easy as possible by
+
+ * Providing snapshot packages regularly for common *NIX platforms and Windows
+ * Providing apt repositories for Debian 6 and Ubuntu
+   10.04+
+ * Providing rpm repositories for latest Fedora releases
 
 == Continuous integration ==
@@ -47 +66 @@
 
 At the moment, there is no coherent set of [http://en.wikipedia.org/wiki/Unit_test unit tests] to spot regressions. One option would be to use [http://cunit.sourceforge.net/index.html CUnit] or similar unit test framework to cover the most commonly used and/or critical codepaths.
-