Opened 9 years ago
Last modified 22 months ago
#626 accepted Bug / Defect
Routing entry not deleted when MAC address moves from client to server
| Reported by: | zewaren | Owned by: | Gert Döring |
|---|---|---|---|
| Priority: | minor | Milestone: | release 2.7 |
| Component: | Networking | Version: | OpenVPN git master branch (Community Ed) |
| Severity: | Not set (select this one, unless your'e a OpenVPN developer) | Keywords: | |
| Cc: |
Description
Context:
A TAP OpenVPN connection is made between two virtual machine hypervisors. The TAP devices are bridged with the VM interfaces, thus creating a L2 bridge between the VMs on different hosts.
A virtual machine (192.168.42.102) is moved from a hypervisor host (node02) to another one (node01).
Initial situation:
.---------------------------------. .---------------------------------. | node01 | | node02 | | OpenVPN server | | OpenVPN client | | | | | | .-----------------------. | | .-----------------------. | | | br0 | | | | br0 | | | | .-------------------. | | | | .-------------------. | | | | | tap0 | | | | | | tap1 | | | | | | 192.168.42.1 |<----------------OpenVPN----------------| 192.168.42.2 | | | | | | de:57:b6:64:0e:c8 | | | | | | 2e:60:ce:c9:63:d1 | | | | | '-------------------' | | | | '-------------------' | | | | | | | | .-------------------. | | | | | | | | | vnet0 | | | | | | | | | | 192.168.42.102 | | | | | | | | | | 52:54:00:de:ad:02 | | | | | | | | | '-------------------' | | | '-----------------------' | | '-----------------------' | '---------------------------------' '---------------------------------'
Final situation:
.---------------------------------. .---------------------------------. | node01 | | node02 | | OpenVPN server | | OpenVPN client | | | | | | .-----------------------. | | .-----------------------. | | | br0 | | | | br0 | | | | .-------------------. | | | | .-------------------. | | | | | tap0 | | | | | | tap1 | | | | | | 192.168.42.1 |<----------------OpenVPN----------------| 192.168.42.2 | | | | | | de:57:b6:64:0e:c8 | | | | | | 2e:60:ce:c9:63:d1 | | | | | '-------------------' | | | | '-------------------' | | | | .-------------------. | | | | | | | | | vnet0 | | | | | | | | | | 192.168.42.102 | | | | | | | | | | 52:54:00:de:ad:02 | | | | | | | | | '-------------------' | | | | | | | '-----------------------' | | '-----------------------' | '---------------------------------' '---------------------------------'
What I do:
Ping 192.168.42.2 (node02) from 192.168.42.102 (the VM).
What happens:
Once the machine is moved from node02 to node01, no ping reply reaches the VM.
What should happen instead:
Ping replies should reach the VM.
OpenVPN version:
OpenVPN 2.3.8 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Aug 4 2015 library versions: OpenSSL 1.0.1e-fips 11 Feb 2013, LZO 2.06 Originally developed by James Yonan Copyright (C) 2002-2010 OpenVPN Technologies, Inc. <sales@openvpn.net> Compile time defines: enable_crypto=yes enable_crypto_ofb_cfb=yes enable_debug=yes enable_def_auth=yes enable_dependency_tracking=no enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=yes enable_fragment=yes enable_http_proxy=yes enable_iproute2=yes enable_libtool_lock=yes enable_lzo=yes enable_lzo_stub=no enable_management=yes enable_multi=yes enable_multihome=yes enable_pam_dlopen=no enable_password_save=yes enable_pedantic=no enable_pf=yes enable_pkcs11=yes enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_pthread=yes enable_selinux=no enable_server=yes enable_shared=yes enable_shared_with_static_runtimes=no enable_small=no enable_socks=yes enable_ssl=yes enable_static=yes enable_strict=no enable_strict_options=no enable_systemd=yes enable_win32_dll=yes enable_x509_alt_username=yes with_crypto_library=openssl with_gnu_ld=yes with_iproute_path=/sbin/ip with_mem_check=no with_plugindir='$(libdir)/openvpn/plugins' with_sysroot=no
Linux version:
CentOS Linux release 7.1.1503 (Core) Linux node02 4.1.12-default #1 SMP Thu Nov 5 16:00:51 CET 2015 x86_64 x86_64 x86_64 GNU/Linux
Server config:
local 198.51.100.7 port 1194 proto udp dev tap0 mode server tls-server ca /etc/openvpn/pki/whatever-ca-crt.pem cert /etc/openvpn/pki/whatever-server-crtchain.pem key /etc/openvpn/pki/whatever-server-key.pem dh /etc/openvpn/pki/whatever-ca-dh.pem tls-auth /etc/openvpn/pki/ta.key 0 server-bridge 192.168.11.1 255.255.255.0 192.168.11.100 192.168.11.200 client-to-client duplicate-cn keepalive 10 120 #comp-lzo max-clients 4 user nobody group nobody persist-key persist-tun status openvpn-status.log log-append openvpn.log verb 9 script-security 2
Client config:
client dev tap1 proto udp remote 198.51.100.7 1194 resolv-retry infinite nobind persist-key persist-tun ca /etc/openvpn/pki/whatever-ca-crt.pem cert /etc/openvpn/pki/whatever-client02-crtchain.pem key /etc/openvpn/pki/whatever-client02-key.pem ns-cert-type server tls-auth /etc/openvpn/pki/ta.key 1 #comp-lzo verb 9 log-append openvpnclient.log script-security 2 keepalive 30 120 float status openvpn-status.log
Here are the log excerpt for a ping packet from the VM the other hypervisor:
Server sends ping packet:
read from TUN/TAP returned 98 GET INST BY VIRT: 2e:60:ce:c9:63:d1 -> OpenVPNClient02/198.51.100.8:45186 via 2e:60:ce:c9:63:d1 OpenVPNClient02/198.51.100.8:45186 TUN READ [98] OpenVPNClient02/198.51.100.8:45186 ENCRYPT FROM: 0000006c 2e60cec9 63d15254 00dead02 08004500 0054f4ea 0000ff01 f104c0a[more...] OpenVPNClient02/198.51.100.8:45186 ENCRYPT TO: 8bc767c2 ebacf0f8 120e8996 092c2d68 675bd569 33189ea8 2aa88961 998fcfb[more...] OpenVPNClient02/198.51.100.8:45186 UDPv4 WRITE [133] to [AF_INET]198.51.100.8:45186: P_DATA_V1 kid=0 DATA c2fe31ee 55bf2043 c24309d1 fef442c3 78d1002d 8bc767c2 ebacf0f8 120e899[more...] OpenVPNClient02/198.51.100.8:45186 UDPv4 write returned 133
Client receives ping packet:
UDPv4 READ [133] from [AF_INET]198.51.100.7:1194: P_DATA_V1 kid=0 DATA c2fe31ee 55bf2043 c24309d1 fef442c3 78d1002d 8bc767c2 ebacf0f8 120e899[more...] DECRYPT TO: 0000006c 2e60cec9 63d15254 00dead02 08004500 0054f4ea 0000ff01 f104c0a[more...] PID_TEST [0] [SSL-0] [112233445566778899>>>>>>>>>>>>>>>>EEEEEEEEEEEEEEEEEEEEEEEEEEEEEE] 0:107 0:108 t=1447061166[0] r=[-2,64,15,0,1] sl=[21,64,64,528] TUN WRITE [98] write to TUN/TAP returned 98
Client sends pong packet:
TUN READ [98] ENCRYPT FROM: 0000006a 525400de ad022e60 cec963d1 08004500 00541a02 00004001 8aeec0a[more...] ENCRYPT TO: 7cc560fa 28a6349d bad24eca 5fee496c b570067a 0b0e3d6e 4f5f518f 4615351[more...] UDPv4 WRITE [133] to [AF_INET]198.51.100.7:1194: P_DATA_V1 kid=0 DATA 31716baa 29bd16a4 68e4a888 a2ffccca 673cbd94 7cc560fa 28a6349d bad24ec[more...] UDPv4 write returned 133
Server receives pong packet and sends it back to client:
OpenVPNClient02/198.51.100.8:45186 UDPv4 READ [133] from [AF_INET]198.51.100.8:45186: P_DATA_V1 kid=0 DATA 31716baa 29bd16a4 68e4a888 a2ffccca 673cbd94 7cc560fa 28a6349d bad24ec[more...] OpenVPNClient02/198.51.100.8:45186 DECRYPT TO: 0000006a 525400de ad022e60 cec963d1 08004500 00541a02 00004001 8aeec0a[more...] OpenVPNClient02/198.51.100.8:45186 GET INST BY VIRT: 52:54:00:de:ad:02 -> OpenVPNClient02/198.51.100.8:45186 via 52:54:00:de:ad:02 OpenVPNClient02/198.51.100.8:45186 ENCRYPT FROM: 0000006d 525400de ad022e60 cec963d1 08004500 00541a02 00004001 8aeec0a[more...] OpenVPNClient02/198.51.100.8:45186 ENCRYPT TO: 5526778c dc9e2db5 64674da6 9d1eab5c c8511d8e bf3ed564 75acdc5c 4dbfa2e[more...] OpenVPNClient02/198.51.100.8:45186 UDPv4 WRITE [133] to [AF_INET]198.51.100.8:45186: P_DATA_V1 kid=0 DATA b70e40a4 2c9e44c2 43545727 7513c0f7 a61148c1 5526778c dc9e2db5 64674da[more...] OpenVPNClient02/198.51.100.8:45186 UDPv4 write returned 133
Client receives pong packet:
UDPv4 read returned 133 UDPv4 READ [133] from [AF_INET]198.51.100.7:1194: P_DATA_V1 kid=0 DATA b70e40a4 2c9e44c2 43545727 7513c0f7 a61148c1 5526778c dc9e2db5 64674da[more...] DECRYPT TO: 0000006d 525400de ad022e60 cec963d1 08004500 00541a02 00004001 8aeec0a[more...] TUN WRITE [98] write to TUN/TAP returned 98
TCPdump on client shows the pong packet coming back:
10:38:31.163434 IP 192.168.42.102 > 192.168.42.2: ICMP echo request, id 48223, seq 806, length 64 10:38:31.163472 IP 192.168.42.2 > 192.168.42.102: ICMP echo reply, id 48223, seq 806, length 64 10:38:31.165248 IP 192.168.42.2 > 192.168.42.102: ICMP echo reply, id 48223, seq 806, length 64
Getting the server's status report indicates that the MAC address of 192.168.42.102 (the VM) is still considered to be reachable through the VPN client while it now should be considered a local address:
OpenVPN CLIENT LIST Updated,Mon Nov 9 10:30:52 2015 Common Name,Real Address,Bytes Received,Bytes Sent,Connected Since OpenVPNClient02,198.51.100.8:45186,63009,96567,Mon Nov 9 10:21:58 2015 ROUTING TABLE Virtual Address,Common Name,Real Address,Last Ref 2e:60:ce:c9:63:d1,OpenVPNClient02,198.51.100.8:45186,Mon Nov 9 10:30:51 2015 52:54:00:de:ad:02,OpenVPNClient02,198.51.100.8:45186,Mon Nov 9 10:30:51 2015 GLOBAL STATS Max bcast/mcast queue length,1 END
Reading the source code indicates that multi_learn_addr() in multi.c does not update routes when the destination addr is local:
https://github.com/OpenVPN/openvpn/blob/ea66a2b5cdb21422139c421b4d3733e1c1c3937e/src/openvpn/multi.c#L1016
Thus, the bad route is kept and is used while it should have been deleted.
When I move the VM between two hypervisors that are both clients of the OpenVPN server (instead of moving it from a client to the server), the condition is respected, the routing table is updated and the network has no problem:
OpenVPNClient02/198.51.100.8:56038 MULTI: Learn: 52:54:00:de:ad:02 -> OpenVPNClient02/198.51.100.8:56038
So, to sum up the diagnostic:
When a MAC address moves from a client to the server, the entry in the routing table is not deleted, and the trafic to the MAC address is sent to the client connection where it was last seen while it should be handled locally.
Should I write and submit a patch?
Attachments (1)
Change History (7)
comment:1 Changed 9 years ago by
Changed 9 years ago by
| Attachment: | patch-multi_process_incoming_tun.patch added |
|---|
comment:2 Changed 9 years ago by
I've attached a patch.
Since it's my first time digging into the code of OpenVPN, I strongly advice core developers to review it.
comment:3 Changed 9 years ago by
| Owner: | set to Gert Döring |
|---|---|
| Status: | new → accepted |
thanks for the patch. Times have been a bit busy, but it is not lost.
comment:6 Changed 22 months ago by
| Milestone: | → release 2.7 |
|---|
*sigh*. I was vaguely aware that we have something here, but was too busy with other aspects. Bumping the milestone to 2.7, need to come back to this.
the bane of l2 mac learning...
Yes, a patch that deletes the MAC address from the routing table (= defaulting to 'tap') when it is seen on the local tap side would be useful, and should be safe enough. A log entry ("MULTI: Learn: $mac -> local tap interface" or something) would be good to understand what is happening, especially if addresses are flip-flop'ing
thanks