Changes between Version 10 and Version 11 of Gigabit_Networks_Linux

Timestamp:

05/02/11 13:09:09 (13 years ago)

Author:

JJK

Comment:

--

Gigabit_Networks_Linux

@@ -4 +4 @@
 == Network setup ==
 For this setup several machines were used, all connected to gigabit switches:
- * two servers running CentOS 5.5 64bit, with an Intel E5440 CPU running @ 2.83GHz; the L2 cache size is 6 MB.
- * a server running CentOS 5.5 64bit, with an Intel X5660 CPU running @ 2.80GHz; the L2 cache size is 12 MB. This CPU has support for the AES-NI instructions.
+ * two servers running CentOS 5.5 64bit, with an Intel Xeon E5440 CPU running @ 2.83GHz; the L2 cache size is 6 MB.
+ * a server running CentOS 5.5 64bit, with an Intel Xeon X5660 CPU running @ 2.80GHz; the L2 cache size is 12 MB. This CPU has support for the AES-NI instructions.
  * a laptop running Fedora 14 64bit, with an Intel i5-560M CPU running @ 2.66GHz; the L2 cache size is 3 MB. This CPU also has support for the AES-NI instructions.
 
@@ -75 +75 @@
 Testing was done similar to the previous tweak
 {{{
-  openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.11 192.168.222.10
+  ./openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.11 192.168.222.10
     --tun-mtu 6000 --fragment 0 --mssfix 0 --cipher aes-256-cbc
 }}}
 and client
 {{{
-  openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.10 192.168.222.11 --remote server
+  ./openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.10 192.168.222.11 --remote server
     --tun-mtu 6000 --fragment 0 --mssfix 0  --cipher aes-256-cbc
 }}}
@@ -101 +101 @@
 
 === Explanation ===
+The AES-NI patch for the OpenSSL 1.0 libraries not only increases performance for AES-NI capable CPUs but also for older CPUs. This can also be seen when running
+{{{
+  openssl speed -evp aes-256-cbc
+}}}
 
-By increasing the MTU size of the tun adapter '''and''' by disabling OpenVPN's internal fragmentation routines the throughput can be increased quite dramatically. The reason behind this is that by feeding larger packets to the OpenSSL encryption and decryption routines the performance will go up. The second advantage of not internally fragmenting packets is that this is left to the operating system and to the kernel network device drivers. For a LAN-based setup this can work, but when handling various types of remote users (road warriors, cable modem users, etc) this is not always a possibility.
+== Using OpenSSL 1.0.0 with AES-NI capable hardware ==
+The real performance gain is expected from the AES-NI capable hardware, such as the Intel Xeon X5660 i5-560M CPUs.
+Using the exact same setup as above, but now using
+{{{
+  engine aesni
+}}}
+and by using 
+{{{
+  ./openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.11 192.168.222.10
+    --tun-mtu 9000 --fragment 0 --mssfix 0 --cipher aes-256-cbc --engine aesni
+}}}
+for the server and 
+{{{
+  ./openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.10 192.168.222.11 --remote server
+    --tun-mtu 9000 --fragment 0 --mssfix 0  --cipher aes-256-cbc --engine aesni
+}}}
+for the client we obtain the following results:
+||              || AES128 || AES256 ||
+||X5660  -> i5-560 || 885 || 878 ||
+||i5-560 -> X5660  || 748 || 543 ||
 
+(Please note that for all measurement a standard deviation of ~5% applies)
 
+Some initial conclusions:
+ * the AES-NI capable server CPU clearly shows a huge performance gain: it is nearly capable of saturating the gigabit network.
+ * it was not possible to run this test with a more capable client CPU. The i5-560M is not capable of keeping up with the X5660, as can be seen from the performance difference between encryption an decryption. In the first entry in the table above the X5660 encrypted all traffic, and the i5-560M decrypted it. The second entry shows the results for the reverse.
+
+== For reference: using no encryption ==
+For reference, the above test was also run with encryption and signing disabled:
+{{{
+  openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.11 192.168.222.10
+    --tun-mtu 9000 --fragment 0 --mssfix 0 --cipher none --auth none
+}}}
+and client
+{{{
+  openvpn --dev tun --proto udp --port 11000 --secret secret.key --ifconfig 192.168.222.10 192.168.222.11 --remote server
+    --tun-mtu 9000 --fragment 0 --mssfix 0  --cipher none --auth none
+}}}
+Now an '''iperf''' result of '''930 Mbps''' is obtained, which shows that performance is not so much limited by the division between kernel-space and user-space processes, but mostly by the encryption and decryption routines as found in the OpenSSL libraries.
+
+== 10 Gigabit networks ==
+The "plaintext" test (i.e. encryption and signing disabled) was repeated on two machines connected to a 10 Gbps switch. 
+
+Again, all results were obtained using 'iperf'.
+
+||         || speed ||
+||"raw"    || 8.8 Gbps ||
+||via tun  || 1.3 Gbps ||
+
+This shows the limits of the kernel-space/user-space division that is present in Linux. The performance of OpenVPN over this network 
+