9 | | For now, see [wiki:Topics-2016-11-07 Topics-2016-11-07]. |
| 9 | Here's an overview of our current, work in progress and proposed QA tools/processes: |
| 10 | |
| 11 | ||Tool/process ||Automated ||In use ||Scope ||Extra requirements ||B.patch||B.merge||A.merge||B.release||B.distro merge|| |
| 12 | ||"make check" ||No[5] ||Yes ||Build ||None ||X || || || || || |
| 13 | ||vagrant ||No ||Needs work ||Build,integration ||Vagrant installed ||X || || || || || |
| 14 | ||unit tests ||Yes ||Needs work ||Regression ||Cmocka installed ||X || || || || || |
| 15 | ||code review ||No ||Yes ||Everything ||None || ||X || || || || |
| 16 | ||travis ||Yes ||Yes ||Build ||Use a !GitHub PR || ||X || || || || |
| 17 | ||patchwork ||Yes ||Proposed ||Build,integration ||Has to be setup || ||X || || || || |
| 18 | ||push scripts ||No ||Yes ||? ||Scripts installed || ||X || || || || |
| 19 | ||buildbot ||Yes ||Yes ||Build,integration ||None || ||(X)![3]||X || || || |
| 20 | ||win builds![1] ||Yes ||Yes ||Build ||None || ||(X)![3]||X || || || |
| 21 | ||win snapshots![1]||Yes ||Yes ||Everything ||Windows || || || ||X || || |
| 22 | ||win testsuite![2]||No![4] ||Yes ||Integration ||Windows || ||(X)![3]||(X)![3]||X || || |
| 23 | ||linux packages ||No![4] ||Yes ||Everything ||Debian OS || ||(X)![3]||(X)![3]||(X)![3] ||X || |
| 24 | |
| 25 | Notes: |
| 26 | |
| 27 | 1. https://github.com/mattock/openvpn-windows-buildtest |
| 28 | 1. https://github.com/mattock/openvpn-windows-test |
| 29 | 1. This tool/process could be adapted to catch errors earlier, at this phase |
| 30 | 1. Could be automated, but requires a fair amount of work; integration with Windows snapshots could be accomplished with [https://chocolatey.org/ Chocolatey]. |
| 31 | 1. "make check" as such does not run automatically at build. However, some of the other CI tools (travis, buildbot) do run it on every commit. |
| 32 | |
| 33 | Key: |
| 34 | |
| 35 | * B.patch = (Typically catches errors) before a patch is published |
| 36 | * B.merge = before merging the patch (to Git "master") |
| 37 | * A.merge = after merging the patch (to Git "master") |
| 38 | * B.release = before making a release |
| 39 | * B.distro merge = before distributions merge the release to their repositories/ports |
| 40 | |
| 41 | ''Scope'' is very high-level here by design. We can improve our QA in two main ways: |
| 42 | |
| 43 | 1. Catch the problems earlier |
| 44 | 1. Make each tool/process catch more problems |
| 45 | |
| 46 | This overview was created originally for community meeting held on [wiki:Topics-2016-11-07 7th November 2016]. |