Development/testing issues

​Beaker test framework

Should we start configuring a dedicated testing framework for OpenVPN? This idea will be presented by mattock.

• Rationale
  • Would allow testing that OpenVPN works on a large number of predefined platforms and configurations
  • Replaces part of the manual testing effort by testing the external behavior of the application automatically
  • Would allow spotting bugs early

• Implementation ideas
  • Beaker could perhaps be made to simulate poor connections for testing purposes. If not, there are a plenty of tools and services available for this purpose (see full chatlog).

• Limitations
  • Only tests for external behavior of the application
  • Does not replace human testing for those classes problems which are difficult/impossible to test programmatically.

Continuous integration server

Should we build a continuous integration (CI) / automated release management server? This idea will be presented by mattock.

• Rationale
  • Would allow spotting build problems early on by building "allmerged" and reporting developers of build problems
  • Would reduce developer frustration
  • Would allow automated packaging of openvpn-testing for various platforms and publishing those on a web server
  • The above would translate to increased use of openvpn-testing, leading to earlier bug reports and would thus make the release process (new code -> acceptance to testing -> acceptance to stable -> release) faster
  • Could also allow centralized automated searching of code problems

• Implementation ideas
  • Developers could be notified of problems using email
  • The OpenVPN project has a license to ​Coverity service. As this service has an API, Coverity code analysis could be integrated to the CI server.

• Limitations
  • Does not replace human testing
  • Only tests the build process (and possibly code problems)

Developer bounties

Should we have a bounty system for writing missing features? Similar systems are in use by several other projects, e.g. freeswitch, pfSense and Funambol. This idea will be presented by ecrist.

• Rationale
  • Would allow users to prioritize feature additions through compensation.
  • Would help achieve greater interest of developers in users' requests and needs.

• Implementation ideas
  • Bounty funds would be setup and maintained by a non-developer (ecrist?) and held. This assures payment upon completion and assures feature is completed before bounty is paid.
  • Multiple users could fund a bounty for intensive feature additions which may warrant higher payouts.
  • Developers setup up accounts and 'claim' tasks
  • After a specific timeout a task becomes available to another user, unless progress can be shown
  • Payment could be divided into multiple phases to motivate developers to maintain the code they've written. For example:
    • 50% payout for a commit to main tree
    • 25% upon release in production (provided bugs are fixed/etc)
    • 25% 6 months after release (provided bugs are fixed/etc)
  • The above would also guarantee the quality of the code
  • Handling monetary transactions
    • Could be handled through an external foundation / organization (e.g. ​http://www.spi-inc.org/projects)
    • We could establish our own foundation for the purpose
  • Licensing and copyright
    • Developer would have to use the BSD license for the bounty features
      • This would allow the project to relicense the code under GPLv2 (while mentioning the original author)
      • This would allow both developer (payee) and payer to use the code any way they wish