Opened 4 years ago

Closed 4 years ago

#348 closed Bug / Defect (fixed)

IPv6 UDP server response may select a different source address than incoming destination preventing formation of the VPN tunnel

Reported by: john7000 Owned by: cron2
Priority: major Milestone: release 2.3.4
Component: Generic / unclassified Version: OpenVPN 2.3.2 (Community Ed)
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords: ipv6
Cc: cron2

Description

Server is Centos 6.3 running OpenVPN 2.3.2. The network is dual stacked and single physical interface (eth0).

OpenVPN is configured for IPv4 and IPv6 both for tunnel end points and transported packets.

To avoid potential routing issues at the client - as it must distinguish between tunnel carrier packets and tunneled packets both destined towards the global IPv6 address of the server - and to aid with firewall security configurations, a secondary global IPv6 address has been configured on the eth0 interface for the VPN tunnel listener.

To prevent the socket connect() function used by various other clients on the server (e.g. ntpd, bind, squid …) from randomly selecting the secondary IPv6 address as the source, the secondary address has been relabelled to a high value (e.g. "ip-f inet6 addrlabel add prefix <secondary-ipv6-addr> label 45"). This causes the socket address selection rule 6 of RFC 3584section 5 to ignore the secondary address whilst a primary address with a lower label value exists.

OpenVPN server UDP responses are seen to come from the primary IPv6 address.

Whilst this secondary address works fine with OpenVPN over TCP over IPv6 (which must reuse the same socket for statefulness) it is not working with UDP over IPv6. Tcpdump and the adjacent router ACLs show that the the response socket used by the server to reply to the client is selecting its own source address and thus using the wrong address.

Any intervening semi-stateful filters such as firewalls and NAT/PAT devices will normally block UDP responses arriving from an arbitrary and different source from the traffic sent outwards. uPNP may overide this behaviour but this is not common or practical in most enterprise networks. The client itself should also be disregarding responses received to the listening UDP socket from arbitrary IP addresses other than the address used to initiate the tunnel connection. Allowing such reponses would open client to potential man-in-the-middle attacks too.

UDP sockets created for the server responses should reuse the original destination address from the client as the source address for the responses.

Change History (8)

comment:1 Changed 4 years ago by john7000

A few issues to clarify

  1. The title is missing a leading letter "I" for IPv6. Is there a way to fix this?
  1. The RFC referenced should be RFC 3484 "Default Address Selection for Internet Protocol version 6 (IPv6)" (not RFC 3584). RFC 5014 may also be relevant.
  1. There appears to be no official meaning to a 'secondary' IP address in the RFCs. The term is used commonly elsewhere such as in the NetworkManager? scripts to distinguish the address(es) from the first added and thus 'primary' address. The 'primary' IPv4 address is often used as a default identifier for a server or in the primary DNS name of the server on the basis that there is usually only one network connection or one management agent address and assumes only one IP address per interface. The address selection criteria may not recognise any such differences.
  1. The label attached to a potential source address just has to be different from that of the destination to deter the use of the 'secondary' address in the selection process. Any label number greater than 4 is sufficient for this purpose.
  1. NAT/PAT normally only applies to IPv4 traffic.
  1. A similar problem may potentially occur with IPv4 where multiple addresses have been assigned to an interface as for example occurs when web servers like IIS or Apache are configured to listen on multiple IP addresses to support multiple domain-named SSL certificates.

comment:2 Changed 4 years ago by john7000

Hi. After a cursory look through the code I suspect there are a number of issues that could be causing the reported behaviour above. Essentially the code appears to be in place to transfer the address information from the inbound UDP traffic received via the recvmesg () call to the response sent via the sendmesg() call.

The first observation is that the code appears to attempt to transfer ipv6 addresses through a simple assignment such as in the following lines of socket.c (from openvpn v2.3.2)

print_link_socket_actual_ex()
#2275: sin6.sin6_addr = act->pi.in6.ipi6_addr;

link_socket_read_udp_posix_recvmsg()
#2726: from->pi.in6.ipi6_addr = pkti6->ipi6_addr;

link_socket_write_udp_posix_sendmsg()
#2839: pkti6->ipi6_addr = to->pi.in6.ipi6_addr;

The problem I see here is that ipi6_addr is a structure containing a union of amongst other formats an array of 16 UINT8 elements. Unless modern C compilers have changed significantly since my programming days I would be surprised if a simple assignment will transfer this much content.

These statements should probably be using something like memcpy() though I could well be wrong.

From /etc/include/linux/ipv6.h
struct in6_pktinfo {

struct in6_addr ipi6_addr;
int ipi6_ifindex;

};

From /etc/include/linux/ipv6.h
struct in6_addr {

union
{

u8 u6_addr8[16];
be16 u6_addr16[8];
be32 u6_addr32[4];

} in6_u;

#define s6_addr in6_u.u6_addr8
#define s6_addr16 in6_u.u6_addr16
#define s6_addr32 in6_u.u6_addr32
};

The second observation is that only one address is transferred and yet we need to conserve both the client-end (transmitter) and server-end (receiver) addresses for use in the sendmesg() function..

According to RFC 3542 section 6.0 on PKTINFO with the flag IPV6_RECVPKTINFO set 'on' should be providing the server ipv6 address for exactly the reason I mentioned previously. What I have not researched yet is how the remote client's IP v6 address is also captured. Clearly this is the case for IPv4 as both ipi_spec_dst and s_addr IPv4 addresses are available.
It appears that IPV6_RECVPKTINFO is being set already so the address from pktinfo should be the servers receiving IPv6 address.

Thirdly, relating to the last observation, link_socket_write_udp_posix_sendmsg() zeros the source address. (Line 2814). This is probably the required behaviour for a client but on a server which may potentially have multiple IPv4 addresses on an interface this should also not be zeroed and should instead be the destination address of the originally received packet. Of course a simple assignment can be used for a 32bit IPv4 address.

Lastly I have not examined the impact of setting the source address of UDP datagrams transmitted by the server on dynamic mobility of the client but given that neither the client nor intervening firewalls are likely to cope easily with mid-session address changes this is probably a far more complex issue. I can imagine the situation occuring for connections from mobile/cell phones as they roam and also for PCs moving between wired and wireless connections. I also wonder whether the openvpn protocols include a keepalive packet type to keep stateful filters and NAT mappings for UDP traffic from timing out when there is no payload to transfer.

If there is likely to be no movement on this bug for some time then I will endeavour during late December 2014 to patch socket.c on my test system to test my theories.

I have not looked for any use of the above mentioned storage structures beyond socket.c as yet nor considered in depth how the sendmesg() may be handling the partial IPv6 address data it is receiving

comment:3 Changed 4 years ago by samuli

  • Cc cron2 added
  • Keywords ipv6 added

comment:4 Changed 4 years ago by cron2

  • Milestone set to release 2.3.3
  • Summary changed from Pv6 UDP server response may select a different source address than incoming destination preventing formation of the VPN tunnel to IPv6 UDP server response may select a different source address than incoming destination preventing formation of the VPN tunnel

There are two other tickets that suggest that --multihome *should* fix this, but there is breakage somewhere in here - #327 and #306.

It's not related to the assignment of structs, that much is clear - C permits assignment of arbitrarily large structs, and will just call memcpy() under the hood.

Could you try with --multihome on the server? That shouldn't zero the source address, but it might still fail.

comment:5 Changed 4 years ago by john7000

--multihome does appear to work after some testing. However I am not yet convinced that the related --nobind command on the client is necessary under these circumstances. In this case there is only one allowed target address for the openVPN server. This is different from situations where the server really does have multiple IP addresses and more than one may be used as a result of round robin multiple DNS entries.

Having examined the code I did wonder why recvmsg()/sendmsg() are only used when multihome is declared with recvfrom()/sendto() being used otherwise. It seems to create unnecessary complexity in the code having both forms of the function calls.

HOWEVER I have found another udp/ipv6 issue. It seems that the fragmentation algorithm does not take into account the larger IPv6 carrier header resulting in oversize UDP/IPv6 packets. I observed that these big UDP packets are then fragmented by the network stack layer creating an additional small packet for the overflow. This packet does not have a UDP header. These raw IPv6 packets then get rejected by firewalls (like Cisco ASAs and routers) on the path. They can be seen to send back ICMPv6 unreachable messages. This causes the VPN communications to fail. Would you like this reported as a new bug? - Now added as #364

Last edited 4 years ago by john7000 (previous) (diff)

comment:6 Changed 4 years ago by cron2

  • Owner set to cron2
  • Status changed from new to assigned

--nobind on the client is not related to --multihome on the server - it's really whether the client will bind() it's socket to a fixed address and/or port, or "use what it will get from the OS".

As for "why is there a switch for --multihome and two different code paths" - I'm not sure, when I joined the development group, this was long there. I assume that since this code in particular is so system-dependent that the original author wanted to keep the "safe" code path for the non-multihoming case...

Anyway, if I understand you right, with --multihome it's working for you? In that case, I'll go ahead and close *this* ticket - the two other referenced tickets will see more comments added to them (Fix to be committed for FreeBSD in #327, pointers to kernel patches for Linux in #306).

comment:7 Changed 4 years ago by john7000

Thanks for the feedback and I am happy for this ticket to be closed.

However my comment about the stated need to use --nobind on the client when using --multihome on the server arises from the following statement in the manual at
https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage
which states:-

--multihome
...

Note: clients connecting to a --multihome server should always use the --nobind option.

Regards

comment:8 Changed 4 years ago by cron2

  • Milestone changed from release 2.3.3 to release 2.3.4
  • Resolution set to fixed
  • Status changed from assigned to closed

Thanks for pointing out this section of the manual. I agree that this is confusing - and I'm not sure why that comment is there, as it doesn't make sense. Actually, half of the text there does not make sense anymore.

I've committed a new version:


.B \-\-multihome
Configure a multi-homed UDP server. This option needs to be used when
a server has more than one IP address (e.g. multiple interfaces, or
secondary IP addresses), and is not using
.B \-\-local
to force binding to one specific address only. This option will
add some extra lookups to the packet path to ensure that the UDP reply
packets are always sent from the address that the client is
talking to. This is not supported on all platforms, and it adds more
processing, so it's not enabled by default.

Note: this option is only relevant for UDP servers.

Note 2: if you do an IPv6+IPv4 dual-stack bind on a Linux machine with
multiple IPv4 address, connections to IPv4 addresses will not work
right on kernels before 3.14, due to missing kernel support for the
IPv4-mapped case.

Note: See TracTickets for help on using tickets.