wiki:WindowsTesting

Introduction

This page outlines the tests we do or should run on Windows. Other pages of interest:

Current test scripts

  • openvpn-windows-buildtest bash scripts
    • Builds an installer on every commit to "release/2.3" and "master" branches
    • Uses openvpn-build
    • Publishes the resulting installers
    • Emails selected developers on build failure
  • openvpn-windows-test Powershell scripts
    • Automated, manually-triggered testing of OpenVPN-GUI, openvpnserv2 and "OpenVPN inside cmd.exe"
    • Tests are done with arbitrary OpenVPN servers (whatever in the configuration directory)
  • run.bat/run2.bat from cron2 (unpublished atm)
    • Runs openvpn inside cmd.exe against the t_client.sh test servers

Usage patterns to test

GUI-based usage (OpenVPN-GUI)

All the basic functionality of OpenVPN GUI should be tested. Connectivity tests have been automated with the openvpn-windows-test.

OpenVPN-GUI communicates with OpenVPN using the management interface, as do all the more recent GUIs. Thus management interface tests (below) will eventually help verify correct operation of OpenVPN-GUI. All the basic functions of OpenVPN-GUI should be tested, but how would these tests be automated? Would it require an automation API in OpenVPN-GUI itself?

OpenVPN as a service (openvpnserv2)

All the basic functionality of openvpnserv2 should be tested. Connectivity tests have been automated with the openvpn-windows-test.

More advanced tests yet to be automated:

  • Ensuring that the service can be enabled / disabled / started / stopped on all platforms
  • Restarting a connection that goes down
  • What happens with broken configuration files? What is the proper behavior?
  • What happens with conflicting configuration files? What is the proper behavior?

Direct use from the command-line

Basic usage from the command-line (cmd.exe) should be tested. This is covered by

Other things that should be tested:

  • Connecting to a VPN as unprivileged user should fail cleanly

Windows-specific command-line tools

netsh

Ensure that the netsh calls that OpenVPN makes have the desired effect. There are no automated tests for this yet.

Windows-specific options

Ensure that various Windows-specific options behave as excepted

ip-win32

  • dynamic (= DHCP)
  • ipapi (= windows API system calls)
  • netsh (= netsh.exe calls)
  • maybe even "adaptive" (= try ipapi, fall back to dynamic)

My minimal run.bat test has cases for that, and it seems --ip-win32 netsh is already unreliable today, even before d12fk's do_ifconfig_after_tun patch.

Management interface

Ensure that management interface commands work as expected, with particular focus on the commands and sequences that a GUI typically uses. These tests should not be run manually, but rather automated directly. Questions:

  • How do we automate the tests?
  • What are the commands and sequences we should test?

Basic connectivity testing

Extend t_client.sh tests to Windows also. Right now: barebones run.bat+run2.bat (see above) and openvpn-windows-test.

Infrastructure needed for the tests

  • t_client server side
  • what about testing "clients on other systems, server on windows"?
Last modified 21 months ago Last modified on 10/17/16 07:00:10