Opened 4 years ago

Last modified 3 years ago

#1310 new Bug / Defect

--tls-crypt-v2-verify causes incorrect client connection status (Potential DDoS)

Reported by: tct Owned by:
Priority: major Milestone: release 2.5.3
Component: Generic / unclassified Version:
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords:
Cc: tct, plaisthos, Gert Döring, Antonio Quartulli, David Sommerseth

Description (last modified by tct)

Notice: --max-clients n is not required. --tls-crypt-v2-verify is the root cause. It was simply that I found it when testing with --max-clients

When using --max-clients n with --tls-crypt-v2-verify, openvpn treats a failed connection from a client as a connected client until Inactivity timeout (--ping-restart) of the failed connection. This can lead to a potential DDOS situation.

Example 1: --max-clients 2 plus --tls-crypt-v2-verify:

A standard server and three standard clients tying to connect. Two of the clients have incorrect credentials to complete the connection. This example uses one valid certificate; one client which has been disabled by --tls-crypt-v2-verify script and one revoked certificate, which is also rejected by --tls-crypt-v2-verify script.

If the two failing clients attempt to connect before the valid client then the valid client will not be allowed to connect. The error in the server log is:
MULTI: new incoming connection would exceed maximum number of clients (2)

The potential DDOS situation:

A malicious client can swamp the server with connection attempts by using --connect-retry 1 2 and multiple client instances.

On a previous run I allowed the malicious clients to successfully block the valid client for over 30 minutes. Another run, made after drafting this report, had two clients block the third for ~20 minutes.

Mitigating actions available:
Do not use --max-clients and --tls-crypt-v2-verify in the same server config.

Example 2: Connections filtered by x509 code for revoked certificates and --client-config-dir file using --disable:

Using the same standard server but without using --tls-crypt-v2-verify, a revoked client certificate attempting a connection is not listed as connected but will continue to attempt connecting, indefinitely. A client which is disabled via a --client-config-dir file --disable option is forcibly shut down by SIGTERM[soft,auth-failure] received, process exiting. Thus, this is not an issue for this server.

Example 1 - Server Log snippets:

Server port is 10127 (management port 12701)
Client c05 is valid from port 12705
Client c06 is revoked from port 12706
Client c08 is disabled from port 12708

Initialization & first malicious client, first connection attempt:

2020-07-29 15:17:14 us=112930 Initialization Sequence Completed

2020-07-29 15:17:16 us=204704 MANAGEMENT: Client connected from [AF_INET]127.0.0.1:12701
2020-07-29 15:17:20 us=38287 MANAGEMENT: CMD 'status 2'
2020-07-29 15:17:28 us=481597 Control Channel: using tls-crypt-v2 key
2020-07-29 15:17:28 us=481675 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:28 us=481699 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:28 us=481716 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:28 us=481730 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:28 us=481747 MULTI: multi_create_instance called
2020-07-29 15:17:28 us=481786 127.0.0.1:12708 Re-using SSL/TLS context
2020-07-29 15:17:28 us=481825 127.0.0.1:12708 tls-crypt-v2 server key: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:28 us=481843 127.0.0.1:12708 tls-crypt-v2 server key: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:28 us=481938 127.0.0.1:12708 Control Channel MTU parms [ L:1621 D:1212 EF:38 EB:0 ET:0 EL:3 ]
2020-07-29 15:17:28 us=481951 127.0.0.1:12708 Data Channel MTU parms [ L:1621 D:1450 EF:121 EB:406 ET:0 EL:3 ]
2020-07-29 15:17:28 us=481980 127.0.0.1:12708 Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
2020-07-29 15:17:28 us=481991 127.0.0.1:12708 Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'

2020-07-29 15:17:28 us=482020 127.0.0.1:12708 TLS: Initial packet from [AF_INET]127.0.0.1:12708, sid=d3d74946 b95f0e8c
2020-07-29 15:17:28 us=482031 127.0.0.1:12708 Control Channel: using tls-crypt-v2 key
2020-07-29 15:17:28 us=482050 127.0.0.1:12708 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:28 us=482065 127.0.0.1:12708 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:28 us=482076 127.0.0.1:12708 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:28 us=482092 127.0.0.1:12708 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication

* TLS-crypt-v2-verify (index) ==> easytls OK ==> custom_group tincantech OK ==> Key age 0 days OK ==> identity OK ==> Client is disabled c08
2020-07-29 15:17:28 us=493190 127.0.0.1:12708 WARNING: Failed running command (--tls-crypt-v2-verify): external program exited with error status: 2
2020-07-29 15:17:28 us=493309 127.0.0.1:12708 TLS CRYPT V2 VERIFY SCRIPT ERROR
2020-07-29 15:17:28 us=493339 127.0.0.1:12708 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]127.0.0.1:12708

Second malicious client, first connection attempt:

2020-07-29 15:17:33 us=600278 127.0.0.1:12706 TLS: Initial packet from [AF_INET]127.0.0.1:12706, sid=05eb2908 bf922bd9

2020-07-29 15:17:33 us=600289 127.0.0.1:12706 Control Channel: using tls-crypt-v2 key
2020-07-29 15:17:33 us=600307 127.0.0.1:12706 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:33 us=600322 127.0.0.1:12706 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:33 us=600334 127.0.0.1:12706 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:33 us=600348 127.0.0.1:12706 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication

* TLS-crypt-v2-verify (index) ==> easytls OK ==> custom_group tincantech OK ==> Key age 0 days OK ==> identity OK ==> Enabled OK ==> Client certificate is revoked: AEABDB92FE2CCF8A9973EF10E32DCAE2 c06
2020-07-29 15:17:33 us=616040 127.0.0.1:12706 WARNING: Failed running command (--tls-crypt-v2-verify): external program exited with error status: 1
2020-07-29 15:17:33 us=616291 127.0.0.1:12706 TLS CRYPT V2 VERIFY SCRIPT ERROR
2020-07-29 15:17:33 us=616344 127.0.0.1:12706 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]127.0.0.1:12706

First connection attempt from valid client:

2020-07-29 15:17:37 us=949072 Control Channel: using tls-crypt-v2 key
2020-07-29 15:17:37 us=949135 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:37 us=949169 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:37 us=949190 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:37 us=949212 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication

2020-07-29 15:17:37 us=949237 MULTI: multi_create_instance called
2020-07-29 15:17:37 us=949285 127.0.0.1:12705 Re-using SSL/TLS context
2020-07-29 15:17:37 us=949322 127.0.0.1:12705 tls-crypt-v2 server key: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:37 us=949340 127.0.0.1:12705 tls-crypt-v2 server key: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:37 us=949434 127.0.0.1:12705 Control Channel MTU parms [ L:1621 D:1212 EF:38 EB:0 ET:0 EL:3 ]
2020-07-29 15:17:37 us=949449 127.0.0.1:12705 Data Channel MTU parms [ L:1621 D:1450 EF:121 EB:406 ET:0 EL:3 ]
2020-07-29 15:17:37 us=949479 127.0.0.1:12705 Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
2020-07-29 15:17:37 us=949491 127.0.0.1:12705 Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'

2020-07-29 15:17:37 us=949503 127.0.0.1:12705 MULTI: new incoming connection would exceed maximum number of clients (2)

2020-07-29 15:17:39 us=33526 Control Channel: using tls-crypt-v2 key
2020-07-29 15:17:39 us=33607 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:39 us=33631 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:39 us=33647 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:39 us=33665 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication

2020-07-29 15:17:39 us=33685 MULTI: multi_create_instance called
2020-07-29 15:17:39 us=33721 127.0.0.1:12705 Re-using SSL/TLS context
2020-07-29 15:17:39 us=33751 127.0.0.1:12705 tls-crypt-v2 server key: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:17:39 us=33769 127.0.0.1:12705 tls-crypt-v2 server key: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:17:39 us=33835 127.0.0.1:12705 Control Channel MTU parms [ L:1621 D:1212 EF:38 EB:0 ET:0 EL:3 ]
2020-07-29 15:17:39 us=33852 127.0.0.1:12705 Data Channel MTU parms [ L:1621 D:1450 EF:121 EB:406 ET:0 EL:3 ]
2020-07-29 15:17:39 us=33890 127.0.0.1:12705 Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
2020-07-29 15:17:39 us=33903 127.0.0.1:12705 Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'

2020-07-29 15:17:39 us=33920 127.0.0.1:12705 MULTI: new incoming connection would exceed maximum number of clients (2)

After killing the malicious clients:

Last failed attempt from valid client ..

2020-07-29 15:21:22 us=425693 127.0.0.1:12705 MULTI: new incoming connection would exceed maximum number of clients (2)

Clients were killed ~30 seconds ago ..

2020-07-29 15:21:39 us=261289 127.0.0.1:12706 [UNDEF] Inactivity timeout (--ping-restart), restarting
2020-07-29 15:21:39 us=261394 127.0.0.1:12706 SIGUSR1[soft,ping-restart] received, client-instance restarting
2020-07-29 15:21:44 us=152170 127.0.0.1:12708 [UNDEF] Inactivity timeout (--ping-restart), restarting
2020-07-29 15:21:44 us=152272 127.0.0.1:12708 SIGUSR1[soft,ping-restart] received, client-instance restarting
2020-07-29 15:21:57 us=671333 Control Channel: using tls-crypt-v2 key
2020-07-29 15:21:57 us=671437 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:21:57 us=671482 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:21:57 us=671509 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:21:57 us=671547 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication

2020-07-29 15:21:57 us=671590 MULTI: multi_create_instance called
2020-07-29 15:21:57 us=671683 127.0.0.1:12705 Re-using SSL/TLS context
2020-07-29 15:21:57 us=671786 127.0.0.1:12705 tls-crypt-v2 server key: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:21:57 us=671820 127.0.0.1:12705 tls-crypt-v2 server key: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:21:57 us=671951 127.0.0.1:12705 Control Channel MTU parms [ L:1621 D:1212 EF:38 EB:0 ET:0 EL:3 ]
2020-07-29 15:21:57 us=671982 127.0.0.1:12705 Data Channel MTU parms [ L:1621 D:1450 EF:121 EB:406 ET:0 EL:3 ]
2020-07-29 15:21:57 us=672051 127.0.0.1:12705 Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
2020-07-29 15:21:57 us=672074 127.0.0.1:12705 Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1541,tun-mtu 1500,proto UDPv4,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'
2020-07-29 15:21:57 us=672135 127.0.0.1:12705 **TLS: Initial packet from** [AF_INET]127.0.0.1:127**05**, sid=b9203aa7 18a979f2
2020-07-29 15:21:57 us=672159 127.0.0.1:12705 Control Channel: using tls-crypt-v2 key
2020-07-29 15:21:57 us=672204 127.0.0.1:12705 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:21:57 us=672236 127.0.0.1:12705 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:21:57 us=672260 127.0.0.1:12705 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
2020-07-29 15:21:57 us=672288 127.0.0.1:12705 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
2020-07-29 15:21:57 us=687208 127.0.0.1:12705 TLS CRYPT V2 VERIFY SCRIPT OK
2020-07-29 15:21:57 us=700779 127.0.0.1:12705 VERIFY OK: depth=1, CN=ChangeMe
2020-07-29 15:21:57 us=700942 127.0.0.1:12705 VERIFY OK: depth=0, CN=c05
2020-07-29 15:21:57 us=701394 127.0.0.1:12705 peer info: IV_VER=2.5_git
2020-07-29 15:21:57 us=701439 127.0.0.1:12705 peer info: IV_PLAT=linux
2020-07-29 15:21:57 us=701456 127.0.0.1:12705 peer info: IV_PROTO=6
2020-07-29 15:21:57 us=701472 127.0.0.1:12705 peer info: IV_NCP=2
2020-07-29 15:21:57 us=701488 127.0.0.1:12705 peer info: IV_CIPHERS=AES-256-GCM:AES-128-GCM
2020-07-29 15:21:57 us=701503 127.0.0.1:12705 peer info: IV_LZ4=1
2020-07-29 15:21:57 us=701518 127.0.0.1:12705 peer info: IV_LZ4v2=1
2020-07-29 15:21:57 us=701533 127.0.0.1:12705 peer info: IV_LZO=1
2020-07-29 15:21:57 us=701549 127.0.0.1:12705 peer info: IV_COMP_STUB=1
2020-07-29 15:21:57 us=701568 127.0.0.1:12705 peer info: IV_COMP_STUBv2=1
2020-07-29 15:21:57 us=701595 127.0.0.1:12705 peer info: IV_TCPNL=1
2020-07-29 15:21:57 us=701609 127.0.0.1:12705 peer info: IV_HWADDR=24:b6:fd:31:bc:ca
2020-07-29 15:21:57 us=701625 127.0.0.1:12705 peer info: IV_SSL=OpenSSL_1.1.1__11_Sep_2018
2020-07-29 15:21:57 us=701638 127.0.0.1:12705 peer info: UV_LONG_STRING=012345678-1-2345678-2-2345678-3-2345678-4-2345678-5-2345678-6-234
2020-07-29 15:21:57 us=702105 127.0.0.1:12705 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 2048 bit RSA
2020-07-29 15:21:57 us=702161 127.0.0.1:12705 [c05] Peer Connection Initiated with [AF_INET]127.0.0.1:12705
2020-07-29 15:21:57 us=702196 c05/127.0.0.1:12705 MULTI_sva: pool returned IPv4=10.127.121.6, IPv6=(Not enabled)
2020-07-29 15:21:57 us=702248 c05/127.0.0.1:12705 MULTI: Learn: 10.127.121.6 -> c05/127.0.0.1:12705
2020-07-29 15:21:57 us=702271 c05/127.0.0.1:12705 MULTI: primary virtual IP for c05/127.0.0.1:12705: 10.127.121.6
2020-07-29 15:21:57 us=702310 c05/127.0.0.1:12705 Data Channel: using negotiated cipher 'AES-256-GCM'
2020-07-29 15:21:57 us=702341 c05/127.0.0.1:12705 Data Channel MTU parms [ L:1549 D:1450 EF:49 EB:406 ET:0 EL:3 ]
2020-07-29 15:21:57 us=702455 c05/127.0.0.1:12705 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2020-07-29 15:21:57 us=702483 c05/127.0.0.1:12705 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
2020-07-29 15:21:57 us=702538 c05/127.0.0.1:12705 SENT CONTROL [c05]: 'PUSH_REPLY,explicit-exit-notify 3,route 10.127.121.1,topology net30,ping 5,ping-restart 30,ifconfig 10.127.121.6 10.127.121.5,peer-id 0,cipher AES-256-GCM' (status=1)

2020-07-29 15:23:23 us=459454 c05/127.0.0.1:12705 SIGTERM[soft,remote-exit] received, client-instance exiting
^C2020-07-29 15:24:30 us=118164 event_wait : Interrupted system call (code=4)
2020-07-29 15:24:30 us=118331 TCP/UDP: Closing socket
2020-07-29 15:24:30 us=118400 net_route_v4_del: 10.127.121.0/28 via 10.127.121.2 dev [NULL] table 0 metric -1
2020-07-29 15:24:30 us=118552 Closing TUN/TAP interface
2020-07-29 15:24:30 us=118577 net_addr_ptp_v4_del: 10.127.121.1 dev tun12701
2020-07-29 15:24:30 us=141686 SIGINT[hard,] received, process exiting

Server status log showing the two malicious client connections:

>INFO:OpenVPN Management Interface Version 3 -- type 'help' for more info
status 2
TITLE,OpenVPN 2.5_git [git:master/20b394746a7a351d] x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Jul 28 2020
TIME,2020-07-29 15:17:20,1596032240
HEADER,CLIENT_LIST,Common Name,Real Address,Virtual Address,Virtual IPv6 Address,Bytes Received,Bytes Sent,Connected Since,Connected Since (time_t),Username,Client ID,Peer ID,Data Channel Cipher
HEADER,ROUTING_TABLE,Virtual Address,Common Name,Real Address,Last Ref,Last Ref (time_t)
GLOBAL_STATS,Max bcast/mcast queue length,0
END
status       
OpenVPN CLIENT LIST
Updated,2020-07-29 15:17:52
Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since
UNDEF,127.0.0.1:12706,1828,0,2020-07-29 15:17:33
UNDEF,127.0.0.1:12708,1828,0,2020-07-29 15:17:28
ROUTING TABLE
Virtual Address,Common Name,Real Address,Last Ref
GLOBAL STATS
Max bcast/mcast queue length,0
END
status 2
TITLE,OpenVPN 2.5_git [git:master/20b394746a7a351d] x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Jul 28 2020
TIME,2020-07-29 15:18:47,1596032327
HEADER,CLIENT_LIST,Common Name,Real Address,Virtual Address,Virtual IPv6 Address,Bytes Received,Bytes Sent,Connected Since,Connected Since (time_t),Username,Client ID,Peer ID,Data Channel Cipher
CLIENT_LIST,UNDEF,127.0.0.1:12706,,,1371,0,2020-07-29 15:18:35,1596032315,UNDEF,3,1,BF-CBC
CLIENT_LIST,UNDEF,127.0.0.1:12708,,,1828,0,2020-07-29 15:18:33,1596032313,UNDEF,2,0,BF-CBC
HEADER,ROUTING_TABLE,Virtual Address,Common Name,Real Address,Last Ref,Last Ref (time_t)
GLOBAL_STATS,Max bcast/mcast queue length,0
END

Change History (10)

comment:1 Changed 4 years ago by tct

Cc: tct added

comment:2 Changed 4 years ago by Steffan Karger

Milestone: release 2.5

This looks like a bug we want to fix before the 2.5.0 release.

Didn't investigate yet.

comment:3 Changed 4 years ago by tct

Further details: This does not require --max-clients.

Server log, last lines:

2020-08-17 09:02:25 us=387878 10.10.201.226:47482 WARNING: Failed running command (--tls-crypt-v2-verify): external program exited with error status: 24
2020-08-17 09:02:25 us=388134 10.10.201.226:47482 TLS CRYPT V2 VERIFY SCRIPT ERROR
2020-08-17 09:02:25 us=388211 10.10.201.226:47482 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]10.10.201.226:47482
2020-08-17 09:04:19 us=936417 10.10.201.226:47482 [UNDEF] Inactivity timeout (--ping-restart), restarting
2020-08-17 09:04:19 us=936542 10.10.201.226:47482 SIGUSR1[soft,ping-restart] received, client-instance restarting

As you can see, --tls-crypt-v2-verify failed and disallowed the connection and then 2 mins later the server --ping-restarted the client. The client was not connected during this time. The server is not using --max-clients. This is purely due to --tls-crypt-v2-verify script

Version 3, edited 4 years ago by tct (previous) (next) (diff)

comment:4 Changed 4 years ago by tct

Description: modified (diff)
Summary: --max-clients with --tls-crypt-v2-verify leads to potential DDOS--tls-crypt-v2-verify leads to potential DDOS

comment:5 Changed 4 years ago by tct

Summary: --tls-crypt-v2-verify leads to potential DDOS--tls-crypt-v2-verify causes incorrect client connection status (Potential DDoS)

comment:6 Changed 4 years ago by tct

Cc: plaisthos Gert Döring Antonio Quartulli David Sommerseth added

Just CC'ing some extra eyes because TLS Crypt V2 is worth fixing.

comment:7 Changed 4 years ago by tct

I did some further testing and can confirm:

  • If a client fails --tls-crypt-v2-verify for any reason then the server sends zero packets in return. (Confirmed by tcpdump)
  • Once a client instance is identified by IP:Port then that same client does not create another instance until the current instance is ping-restart-ed by the server:

EG (Server has --ping-restart 30):

2020-09-03 19:48:18 us=223163 MULTI: multi_create_instance called
2020-09-03 19:48:18 us=223301 127.0.0.1:12705 Re-using SSL/TLS context

...

2020-09-03 19:48:18 us=235997 127.0.0.1:12705 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]127.0.0.1:12705
2020-09-03 19:48:20 us=459204 127.0.0.1:12705 TLS: Initial packet from [AF_INET]127.0.0.1:12705, sid=8012bbdc ac413169

...

2020-09-03 19:48:20 us=473348 127.0.0.1:12705 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]127.0.0.1:12705
2020-09-03 19:48:24 us=932052 127.0.0.1:12705 TLS: Initial packet from [AF_INET]127.0.0.1:12705, sid=8012bbdc ac413169

...

2020-09-03 19:48:24 us=943476 127.0.0.1:12705 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]127.0.0.1:12705
2020-09-03 19:48:32 us=413462 127.0.0.1:12705 TLS: Initial packet from [AF_INET]127.0.0.1:12705, sid=8012bbdc ac413169

...

2020-09-03 19:48:32 us=425321 127.0.0.1:12705 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]127.0.0.1:12705
2020-09-03 19:48:48 us=109891 127.0.0.1:12705 [UNDEF] Inactivity timeout (--ping-restart), restarting
2020-09-03 19:48:48 us=110004 127.0.0.1:12705 SIGUSR1[soft,ping-restart] received, client-instance restarting

...

2020-09-03 19:48:48 us=904115 MULTI: multi_create_instance called
2020-09-03 19:48:48 us=904297 127.0.0.1:12705 Re-using SSL/TLS context

comment:8 Changed 3 years ago by tct

I did a little more testing..

Running 12 clients which all fail at --tls-crypt-v2-verify with this configuration:

Server:

keepalive 10 60 # Standard default

Client:

connect-retry 2 2 # To hammer the server
connect-timeout 2

Log failure (Note: Port):

2021-03-17 17:42:02 us=536375 10.10.201.226:59142 TLS CRYPT V2 VERIFY SCRIPT ERROR
2021-03-17 17:42:02 us=536416 10.10.201.226:59142 TLS Error: can not extract tls-crypt-v2 client key from [AF_INET]10.10.201.226:59142

The server then has the client in a partially connected state until:

Log timeout (Note: Port):

2021-03-17 17:44:02 us=665301 10.10.201.226:59142 [UNDEF] Inactivity timeout (--ping-restart), restarting
2021-03-17 17:44:02 us=665416 10.10.201.226:59142 SIGUSR1[soft,ping-restart] received, client-instance restarting

I let the test run for a couple of hours and found that memory usage tops out at what-ever usage is required to maintain these partial connections.

Using top I found that on my system openvpn server topped out at around 75MB.

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
1122173 root      20   0   75580  72844   6272 S   6.0   0.9   5:07.45 openvpn

Inspecting the server log, I would estimate concurrent partial connections peaked at around 720. (12 clients connecting 30 times per minute in a 2 minute window)

I guess this means that, while it's probably not the way you want to handle failed --tls-crypt-v2-verify connections, openvpn does manage the situation in a robust manner. IE. Memory is not unexpectedly consumed.

Last edited 3 years ago by tct (previous) (diff)

comment:9 Changed 3 years ago by Gert Döring

Milestone: release 2.5release 2.5.3

comment:10 Changed 3 years ago by tct

In summary:

--tls-crypt-v2-verify failure appears to invoke code that ought not be started. Once failed, the client can/should be instantaneously forgotten, barring some form of timeout/back-off, as yet to be coded.

Note: See TracTickets for help on using tickets.