Development/testing issues

Beaker test framework

Should we start configuring a dedicated testing framework for OpenVPN? This idea will be presented by mattock.

  • Rationale
    • Would allow testing that OpenVPN works on a large number of predefined platforms and configurations
    • Replaces part of the manual testing effort by testing the external behavior of the application automatically
    • Would allow spotting bugs early
  • Implementation ideas
    • Beaker could perhaps be made to simulate poor connections for testing purposes. If not, there are a plenty of tools and services available for this purpose (see full chatlog).
  • Limitations
    • Only tests for external behavior of the application
    • Does not replace human testing for those classes problems which are difficult/impossible to test programmatically.

Continuous integration server

Should we build a continuous integration (CI) / automated release management server? This idea will be presented by mattock.

  • Rationale
    • Would allow spotting build problems early on by building "allmerged" and reporting developers of build problems
    • Would reduce developer frustration
    • Would allow automated packaging of openvpn-testing for various platforms and publishing those on a web server
    • The above would translate to increased use of openvpn-testing, leading to earlier bug reports and would thus make the release process (new code -> acceptance to testing -> acceptance to stable -> release) faster
    • Could also allow centralized automated searching of code problems
  • Implementation ideas
    • Developers could be notified of problems using email
    • The OpenVPN project has a license to Coverity service. As this service has an API, Coverity code analysis could be integrated to the CI server.
  • Limitations
    • Does not replace human testing
    • Only tests the build process (and possibly code problems)

Developer bounties

Should we have a bounty system for writing missing features? Similar systems are in use by several other projects, e.g. freeswitch, pfSense and Funambol. This idea will be presented by ecrist.

  • Rationale
    • Would allow users to prioritize feature additions through compensation.
    • Would help achieve greater interest of developers in users' requests and needs.
  • Implementation ideas
    • Bounty funds would be setup and maintained by a non-developer (ecrist?) and held. This assures payment upon completion and assures feature is completed before bounty is paid.
    • Multiple users could fund a bounty for intensive feature additions which may warrant higher payouts.
    • Developers setup up accounts and 'claim' tasks
    • After a specific timeout a task becomes available to another user, unless progress can be shown
    • Payment could be divided into multiple phases to motivate developers to maintain the code they've written. For example:
      • 50% payout for a commit to main tree
      • 25% upon release in production (provided bugs are fixed/etc)
      • 25% 6 months after release (provided bugs are fixed/etc)
    • The above would also guarantee the quality of the code
    • Handling monetary transactions
    • Licensing and copyright
      • Developer would have to use the BSD license for the bounty features
        • This would allow the project to relicense the code under GPLv2 (while mentioning the original author)
        • This would allow both developer (payee) and payer to use the code any way they wish
Last modified 8 years ago Last modified on 05/21/10 07:51:41