Improve TCP-over-TCP performance

[Samuli Seppänen]

Tunneling TCP over TCP without performance penalty is difficult. However, nothing is stopping us from trying.

[Samuli Seppänen]

Wireshark pcap file for a direct (non-openvpn) connection, part 1

[Samuli Seppänen]

Wireshark pcap file for a direct (non-openvpn) connection, part 2

[Samuli Seppänen]

Wireshark pcap file for a TCP-over-TCP via OpenVPN connection

[Samuli Seppänen]

Graph of a direct TCP connection

[Samuli Seppänen]

Another graph of a direct TCP connection

[Samuli Seppänen]

Graph of a TCP-over-TCP connection via OpenVPN

[Samuli Seppänen]

Another graph of a TCP-over-TCP connection via OpenVPN

[Samuli Seppänen]

Tunneling TCP traffic inside an OpenVPN TCP tunnel causes performance to drop significantly if the connection is unreliable (e.g. Internet). Unfortunately TCP-over-TCP is a necessity when the OpenVPN connection has to go through a proxy. The OpenVPN*.png pictures show a typical(?) throughput patterns for TCP traffic going through an TCP-based OpenVPN tunnel. As can be seen, at times throughput is good, but it often drops very low. This behavior apparently caused by the duplicated reliability layer of the nested TCP connections.

These issues have been discussed before on the mailing lists:

• ​http://article.gmane.org/gmane.network.openvpn.user/640
• ​http://thread.gmane.org/gmane.network.openvpn.user/29601

Any help with analyzing the logs or optimizing network settings to maximize OpenVPN TCP-over-TCP performance is most welcome!

[arrmo]

Hi,

Here are some results, between two machines in my network. The results noted as "direct" are exactly that - a link between the two machines, without OpenVPN running. The "OpenVPN" results are then of course with OpenVPN running (between the same two machines, but traffic over OpenVPN).

Note the significant impact that the TCP Window size has on OpenVPN performance ... any thoughts?

Thanks!

[arrmo]

iperf results (arrmo), 1/4

[arrmo]

iperf results (arrmo), 2/4

[arrmo]

iperf results (arrmo), 3/4

[arrmo]

iperf results (arrmo), 4/4

[Gert Döring]

So, has anyone in network research found a good approach to make tcp-over-tcp fast?

It works for OpenSSH tunneled stuff (-L/-R/-D) because they don't actually do tcp-over-tcp but terminate the connection on one end and build a new one on the other side.

We could do tcp-local-ack, but that would require fairly large new code to be written...

[L29Ah]

Is there a VPN implementation that does this?

[David Sommerseth]

There is really no good solution to this as we can imagine now.

[wangyucn]

So, has anyone in network research found a good approach to make tcp-over-tcp fast?

@cron2 @L29Ah

I have a solution which can kind of make tcp-over-tcp fast.I implemented a custom TCP protocol(we can call it FakeTCP)at user space by using raw socket.It simulates 3-way handshake and seq/ack of standard TCP,but it allows out-of-order delivery and there is no re-transmit/congestion control.Firewalls and routers will just regard it as normal TCP.

It can pass through NAT(as UDP or normal TCP does),works very stably.By using this FakeTCP protocol,the performance can be as good as UDP.Its useful when UDP is blocked or being throttled.

Check this URL for more detail:
​https://github.com/wangyu-/udp2raw-tunnel#tunneling-any-traffic-via-raw-traffic-by-using-udp2raw-openvpn

Its currently an external program,but its also possible to integrate the idea into OpenVPN.

[wangyucn]

performance test of faketcp

[wangyucn]

how faketcp works with openvpn

[Gert Döring]

Thanks, this is interesting.

We are not going to implement it, though. OpenVPN over TCP is needed when you have nasty firewalls out there that do sequence number checking and all that - and then your FakeTCP is not going to work either. If you have no firewalls in the way, OpenVPN over UDP works perfectly well (including "through NAT").

It might be interesting to eventually implement openvpn over QUIC (HTTP/2), which is UDP based, but "firewalls are learning to accept it, because Google is pushing it". But that's a different topic.

[wangyucn]

We are not going to implement it, though

Thats okay.UPDATE:I just cant agree with the reason you gave.It seems like you rejected a general feature for a rare circumstance.

OpenVPN over TCP is needed when you have nasty firewalls out there that do sequence number checking and all that - and then your FakeTCP is not going to work either.

Not 100% agree,OpenVPN over TCP is needed whenever UDP is blocked or being throttled.To have a UDP blocked firewall doesnt mean to have a nasty tcp firewall at same time.Having an extremely nasty tcp firewall is another problem,and its rare.

In the circumstance of just UDP is blocked or being throttled,a normal TCP has performance issue while FakeTCP doesnt.

This method solves most of the troubles when UDP is not avaliable,it only fails when there is also an extremely nasty tcp firewall as you mentioned.

Actually this FakeTCP has sequence number simulating.If firewalls want to detect it,they have to trace packet re-transmit or congestion control,it will be costly.We havent got any report of being detected so far.Have to admit its possible to be detect in furture in theory,but not all firewalls can do or want to do such kind of packet inspection.

If you have no firewalls in the way, OpenVPN over UDP works perfectly well

Partially right.UDP is the best and straightforward way when there are no firewalls and UDP is well supported.

Under some circumstance,there is no firewall at all,but UDP is not well supported(for example by NAT device),you will have to use TCP.

We are not going to implement it, though

Fighting against firewalls or dealing with UDP unstable environment maybe kind of beyond OpenVPN 's design purpose if developers think so(design purpose is developers' personal decision),I fully understand,if this feature cant be implemented if this is the reason why it cant be implemented.