Blame

719005 Samuli Seppänen 2025-02-28 16:22:36 1
# Server side testing 
2
3
## OpenVPN server implementations
4
5
* OpenVPN 2.x server
6
* Linux distros
7
* *BSD
8
* Windows (yes!)
9
* Cloudconnexa: https://openvpn.net/cloud-vpn/
10
* Developed by OpenVPN Inc.
11
* Previously known as OpenVPN Cloud
12
* Access Server: https://openvpn.net/access-server/
13
* Developed by OpenVPN Inc.
14
* Based on OpenVPN 2.x
15
* Azure VPN Gateway: https://learn.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-about-vpngateways
16
* Developed by Microsoft
17
* Reimplementation of the OpenVPN protocol
18
* TCP only
19
* AWS Client VPN: https://docs.aws.amazon.com/vpn/latest/clientvpn-user/client-vpn-user-what-is.html
20
* Managed service by AWS
21
* Technical details unknown. May be just using OpenVPN 2.x under the hood.
22
* Softether
23
* Open source. Minimal OpenVPN implementation lacks most newer features.
24
25
## Server-side testing in July 2024
26
27
### The challenges
28
29
* "A 2.6 client on FreeBSD will behave differently than a 2.6 client on Linux with DCO than a 2.6 client on Linux without DCO. and all this might behave different if the server is 2.3 or 2.4." (from cron2 in IRC)
30
* "Getting proper server instances with arbitrary openvpn versions running is hard, as options have changed quite a bit (so my current server instances won't run on 2.5 due to config incompatibilities - and 2.5 configs won't work with master servers due to certificate shenanigans and such) (from cron2 on IRC)
31
* "AFAIK DCO on FreeBSD has two drawbacks: 1. Running with UID != 0 breaks it (--user option). 2. "multihome" is not correctly supported. (from mzar on IRC)
32
* FreeBSD 14 buildbot worker is missing
33
34
### What is being tested?
35
36
* Client tests (t_client.sh)
37
* Buildbot launches builds for each "default" build type (openssl, mbedtls) against a set of pre-configured t_client test servers running a version OpenVPN 2.x
38
* This includes tests that use HTTP and SOCKS proxies
39
* This includes Linux DCO tests
40
* Server tests (t_server_null.sh)
41
* Buildbot launches multiple server instances and launches multiple clients instances against the servers
42
* Both servers and clients can have arbitrary configurations
43
* Detailed description of t_server_null.sh below
44
* Daily scheduled t_client.sh tests of 2.2, 2.3, 2.4, 2.5 and master clients against "Git" master servers
45
* These t_client tests for "testing server functionality" look different from the "test client" - the server provides --client-connect scripts, which can be made to succeed or fail, synchronously or asynchronously, by sending UV_ variables over. This is important to test the *server*, but for testing clients it would be just a waste of time ("the client does the same thing again and again")
46
47
### What is not being tested?
48
49
* Connecting to some arguably important non-FOSS / non-standard server-side implementations (see above)
50
* Connecting to "just compiled" Git "master" server using "oldstable" (release/2.5) or "stable" (release/2.6) clients
51
* Many other client/server combinations
52
* Servers built from "oldstable" (release/2.5) or "stable" (release/2.6) branches
53
* Linux DCO on the client or server?
54
55
### Known issues
56
57
* Buildbot can't run P2P tests
58
59
### Internal OpenVPN testing at OpenVPN Inc
60
61
OpenVPN Inc. has a fairly large client test suite. It consists of OpenVPN clients with a number of different profiles and supporting OpenVPN server side. The test suite did, and maybe still does, run manually. Some of this work could potentially be integrated with community testing efforts and automated.
62
63
## Next steps
64
65
We can store "stuff" in https://github.com/OpenVPN/openvpn-tests so that other people can get access to it.
66
67
1. t_server_null.sh: test older (2.5, 2.6) clients against the "just compiled" Git "master" server
68
1. add ARM64-based buildbot workers to improve server (and client) coverage
69
1. convert Gert's testing framework to --dev null for testing basic client functionalities
70
1. implement lightweight client using --dev null (with lwIP). Client should be able to reply to pings and possibly to HTTP requests
71
1. extend t_client to work with aforementioned mechanism: ping enters the tun and goes to client
72
1. later: come up with more complex client tests (one client is connected, the next one reconnects multiple times, etc..)
73
1. later: do we want to test mgmt interface?
74
1. later later: replace t_client with other test drivers
75
76
77
## t_server_null.sh
78
79
You can configure OpenVPN to use a "null" device. When you use a "null" device OpenVPN will not attempt to configure network interfaces or routing after starting up (server) or connecting (client). You don't even have to be root or use sudo this way, because no privileged operations are needed. A "--dev null" client can still be used to connect to an OpenVPN server and to verify that the connection was established correctly. Using "--dev null" allows running multiple, quick client <-> server integration tests on localhost without any dedicated server infrastructure while maintaining very high reliability due to traffic never leaving the loopback interface.
80
81
The tests run the "just compiled openvpn" as a server in various configurations. The clients are currently "just compiled openvpn" as well, but they can be any other version that is available locally. The [current implementation](https://github.com/OpenVPN/openvpn/blob/master/tests/t_server_null.sh) works like this:
82
83
1. **make check**
84
1. **t_server_null.sh**
85
1. **t_server_null_server.sh**
86
* Launches the compiled OpenVPN server instances as root (if necessary with sudo or doas) in the background
87
* OpenVPN servers exit when all clients have disconnected from all servers
88
1. **t_server_null_client.sh**
89
* Launches each individual client with --dev null in the background
90
* Each client kills itself after some delay using an "--up" script
91
92
The features:
93
94
* Parallelized for fairly high performance
95
* Operating system agnostic
96
* Tested on the following platforms:
97
* Fedora Linux 38-40 and "rawhide"
98
* Ubuntu 20.04, 22.04 and 24.04
99
* Debian 10-12 and "unstable"
100
* OpenSuSE Leap 15
101
* Alpine Linux
102
* Arch Linux
103
* FreeBSD 12-14
104
* OpenBSD 6.8 and 7.5
105
* NetBSD 8.1 and 10.0
106
* POSIX shell compliant
107
* Tested with the following shells:
108
* Bash
109
* Dash
110
* Yash
111
* Ksh
112
* FreeBSD "/bin/sh" (a bourne shell variant)
113
* NetBSD "/bin/sh" (a bourne shell variant)
114
* Near 100% reliable (so far no test failures)
115
* Uses the sample certificates and keys
116
* Supports running servers directly as root and with sudo/doas
117
* Supports using different OpenVPN client versions
118
* The "current" (just compiled) version
119
* Any other OpenVPN versions that is present on the filesystem
120
* Support testing for success as well as failure
121
* Test cases (client configurations) and server setups (server configurations) are stored in a configuration file, i.e. data and code have been separated
122
* Configuration file format is nearly identical to t_client.rc configuration
123
* Supports a set of default tests, overriding default test settings and adding local tests