Changes between Version 32 and Version 33 of OpenVPN_QA


Ignore:
Timestamp:
11/07/16 20:53:25 (7 years ago)
Author:
Samuli Seppänen
Comment:

Add overview of current QA table

Legend:

Unmodified
Added
Removed
Modified
  • OpenVPN_QA

    v32 v33  
    77= Overview of current tools and processes =
    88
    9 For now, see [wiki:Topics-2016-11-07 Topics-2016-11-07].
     9Here's an overview of our current, work in progress and proposed QA tools/processes:
     10
     11||Tool/process     ||Automated ||In use     ||Scope              ||Extra requirements ||B.patch||B.merge||A.merge||B.release||B.distro merge||
     12||"make check"     ||No[5]     ||Yes        ||Build              ||None               ||X      ||       ||       ||         ||              ||
     13||vagrant          ||No        ||Needs work ||Build,integration  ||Vagrant installed  ||X      ||       ||       ||         ||              ||
     14||unit tests       ||Yes       ||Needs work ||Regression         ||Cmocka installed   ||X      ||       ||       ||         ||              ||
     15||code review      ||No        ||Yes        ||Everything         ||None               ||       ||X      ||       ||         ||              ||
     16||travis           ||Yes       ||Yes        ||Build              ||Use a !GitHub PR   ||       ||X      ||       ||         ||              ||
     17||patchwork        ||Yes       ||Proposed   ||Build,integration  ||Has to be setup    ||       ||X      ||       ||         ||              ||
     18||push scripts     ||No        ||Yes        ||?                  ||Scripts installed  ||       ||X      ||       ||         ||              ||
     19||buildbot         ||Yes       ||Yes        ||Build,integration  ||None               ||       ||(X)![3]||X      ||         ||              ||
     20||win builds![1]   ||Yes       ||Yes        ||Build              ||None               ||       ||(X)![3]||X      ||         ||              ||
     21||win snapshots![1]||Yes       ||Yes        ||Everything         ||Windows            ||       ||       ||       ||X        ||              ||
     22||win testsuite![2]||No![4]    ||Yes        ||Integration        ||Windows            ||       ||(X)![3]||(X)![3]||X        ||              ||
     23||linux packages   ||No![4]    ||Yes        ||Everything         ||Debian OS          ||       ||(X)![3]||(X)![3]||(X)![3]  ||X             ||
     24
     25Notes:
     26
     271. https://github.com/mattock/openvpn-windows-buildtest
     281. https://github.com/mattock/openvpn-windows-test
     291. This tool/process could be adapted to catch errors earlier, at this phase
     301. Could be automated, but requires a fair amount of work; integration with Windows snapshots could be accomplished with [https://chocolatey.org/ Chocolatey].
     311. "make check" as such does not run automatically at build. However, some of the other CI tools (travis, buildbot) do run it on every commit.
     32
     33Key:
     34
     35* B.patch = (Typically catches errors) before a patch is published
     36* B.merge = before merging the patch (to Git "master")
     37* A.merge = after merging the patch (to Git "master")
     38* B.release = before making a release
     39* B.distro merge = before distributions merge the release to their repositories/ports
     40
     41''Scope'' is very high-level here by design. We can improve our QA in two main ways:
     42
     431. Catch the problems earlier
     441. Make each tool/process catch more problems
     45
     46This overview was created originally for community meeting held on [wiki:Topics-2016-11-07 7th November 2016].
    1047
    1148= Static testing =