Introduction
This page outlines the tests we do or should run on Windows. Other pages of interest:
Current test scripts
- openvpn-windows-buildtest bash scripts
- Builds an installer on every commit to "release/2.3" and "master" branches
- Uses openvpn-build
- Publishes the resulting installers
- Emails selected developers on build failure
- openvpn-windows-test Powershell scripts
- Automated, manually-triggered testing of OpenVPN-GUI, openvpnserv2 and "OpenVPN inside cmd.exe"
- Tests are done with arbitrary OpenVPN servers (whatever in the configuration directory)
- run.bat/run2.bat from cron2 (unpublished atm)
- Runs openvpn inside cmd.exe against the t_client.sh test servers
Usage patterns to test
GUI-based usage (OpenVPN-GUI)
All the basic functionality of OpenVPN GUI should be tested. Connectivity tests have been automated with the openvpn-windows-test.
OpenVPN-GUI communicates with OpenVPN using the management interface, as do all the more recent GUIs. Thus management interface tests (below) will eventually help verify correct operation of OpenVPN-GUI. All the basic functions of OpenVPN-GUI should be tested, but how would these tests be automated? Would it require an automation API in OpenVPN-GUI itself?
OpenVPN as a service (openvpnserv2)
All the basic functionality of openvpnserv2 should be tested. Connectivity tests have been automated with the openvpn-windows-test.
More advanced tests yet to be automated:
- Ensuring that the service can be enabled / disabled / started / stopped on all platforms
- Restarting a connection that goes down
- What happens with broken configuration files? What is the proper behavior?
- What happens with conflicting configuration files? What is the proper behavior?
Direct use from the command-line
Basic usage from the command-line (cmd.exe) should be tested. This is covered by
- cron2's run.bat/run2.bat script
- openvpn-windows-test
Other things that should be tested:
- Connecting to a VPN as unprivileged user should fail cleanly
Windows-specific command-line tools
netsh
Ensure that the netsh calls that OpenVPN makes have the desired effect. There are no automated tests for this yet.
Windows-specific options
Ensure that various Windows-specific options behave as excepted
ip-win32
- dynamic (= DHCP)
- ipapi (= windows API system calls)
- netsh (= netsh.exe calls)
- maybe even "adaptive" (= try ipapi, fall back to dynamic)
My minimal run.bat test has cases for that, and it seems --ip-win32 netsh
is already unreliable today, even before d12fk's do_ifconfig_after_tun patch.
Management interface
Ensure that management interface commands work as expected, with particular focus on the commands and sequences that a GUI typically uses. These tests should not be run manually, but rather automated directly. Questions:
- How do we automate the tests?
- What are the commands and sequences we should test?
Basic connectivity testing
Extend t_client.sh tests to Windows also. Right now: barebones run.bat+run2.bat (see above) and openvpn-windows-test.
Infrastructure needed for the tests
- t_client server side
- what about testing "clients on other systems, server on windows"?