Changes between Version 3 and Version 4 of PerformanceTesting


Ignore:
Timestamp:
10/04/11 11:47:50 (12 years ago)
Author:
Samuli Seppänen
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PerformanceTesting

    v3 v4  
    99The tests have several goals:
    1010
    11  * Get an idea of OpenVPN performance with given parameters
     11 * Get an idea of OpenVPN performance with any given parameters
    1212 * Identify performance bottlenecks and fix the underlying issues insofar as possible
    1313 * Identify non-linear resource comsumption growth, e.g. find out if 100 clients consumes (significantly) more resources than that of 10 clients (multiplied by ten)
     14 * Detect and fix abnormal behavior (e.g. partially configured clients) during peak load periods
    1415
    1516= Prequisites =
    1617
    17 Identification of performance bottlenecks requires plotting resource consumption, e.g. ''memory usage'' and ''processor load''. This, in turn may require selective adding of new debugging code, so that excessively verbose, generic logging does not degrade performance and thus produce biased results. Maximum performance tests don't have this requirement, as they only need minimal logging.
     18Identification of performance bottlenecks requires plotting resource consumption, e.g. ''memory usage'' and ''processor load''. This, in turn may require selective adding of new debugging code, so that excessively verbose, generic logging does not degrade performance and thus produce biased results. Maximum performance tests don't have this requirement, as they only need minimal logging. In addition, spotting and debugging configuration errors requires collecting information about OpenVPN's state on the client side.
    1819
    1920There are further requirements, depending on the test environments being used; see below for details.
    2021
    21 = Throughput tests =
     22= Test cases =
     23
     24Here we should have a high-level list of test cases. Technical details (e.g. OpenVPN client configuration files) can also be attached. Mattock can then convert these descriptions into Fabric code which makes the magic happen.
     25
     26== Throughput tests ==
    2227
    2328There are a number of factors affecting throughput:
     
    3439There are a few specific questions that need an answer:
    3540
    36  * Throughput using IPv4 code
    37  * Throughput using IPv6 code
    38  * Throughput using OpenSSL
    39  * Throughput using PolarSSL
     41 * Maximum throughput using various parameters
    4042 * MTU settings and throughput
    4143 * How does (excessive) packet loss affect performance?
     
    4446 * Other questions, which?
    4547
     48Below is a list of proposed test cases. All bandwidth (BW) numbers are in MB/s (megabytes per second).
     49
     50||'''Test name'''||'''Min clients'''||'''Max clients'''||'''BW/client'''||'''Bandwidth rise'''||'''Description'''||'''Measures'''||
     51||TP-1||1||10||100||Gradual||Few clients with high bandwidth||Max average throughput per second||
     52||TP-2||10||100||10||Gradual||Many clients with medium bandwidth||Max average throughput per second||
     53
     54Each test will be repeated for the following combinations:
     55
     56||'''IPv4'''||'''IPv6'''||'''OpenSSL'''||'''PolarSSL'''||'''Test results'''||
     57||X|| ||X|| ||n MB/s||
     58||X|| || ||X||n MB/s||
     59|| ||X||X|| ||n MB/s||
     60|| ||X|| ||X||n MB/s||
     61
     62
    4663Obviously the goal is to identify poorly performing parts of the code (if any) and improve them.
    4764
    48 = Connection tests =
     65== Connection tests ==
    4966
    5067Connection tests are aimed at identifying bottlenecks during spikes of (re)connection initiations, e.g. when 100 clients try to connect to the server simultaneously.
     
    126143
    127144The ''README'' file contains everything you need to know. At least initially patches should be sent to samuli at openvpn and net, or to ''mattock'' who hangs around at #openvpn-devel on the Freenode IRC network (irc.freenode.net).
    128 
    129 = Test cases =
    130 
    131 Here we should have a high-level list of test cases. Technical details (e.g. OpenVPN client configuration files) can also be attached. Mattock can then convert these descriptions into Fabric code which makes the magic happen.