Opened 11 months ago

Closed 4 weeks ago

#1226 closed Bug / Defect (fixed)

spurious failures with IPv6 over TAP on FreeBSD 12 (ND6_IFF_IFDISABLED)

Reported by: Gert Döring Owned by: Gert Döring
Priority: major Milestone: release 2.5
Component: Networking Version: OpenVPN git master branch (Community Ed)
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords: freebsd tap IFDISABLED
Cc:

Description

Setting up a new testbed with the clients run on FreeBSD 12 (12.0-RELEASE-p11) I ran into spurious IPv6 failures in the TAP tests.

In all those cases, the tap interface config showed

tap0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:2d:62:5f:00
        inet6 fe80::2bd:2dff:fe62:5f00%tap0 prefixlen 64 tentative scopeid 0x3
        inet6 fd00:abcd:204:4::1001 prefixlen 64 tentative
        inet 10.204.4.3 netmask 0xffffff00 broadcast 10.204.4.255
        groups: tap
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        Opened by PID 65794

all IPv6 addresses are flagged as "tentative" (and stay so), and the "IFDISABLED" flag is set. In all cases that work, this flag is not set.

This is not dependent on OpenVPN version (it happens with 2.4.8 and git master) nor config (if you run the same test 8 times in a row, most times it will succeed and sometimes it fails).

It is very likely timing dependent, as it seems to happen more often on a FreeBSD 11.3 VM which is run on a different (faster) host...

The main intention of this ticket is "record the fact so it can be found with search engines". I think it's a FreeBSD bug, but have no time right now to find people that can look into it.

One possible workaround is to use an up script

script-security 2
up /usr/local/etc/openvpn/inet6-no-ifdisabled.sh

which contains

#!/bin/sh
IF=$1

( sleep 2 ; ifconfig $IF inet6 -ifdisabled ) &

(the sleep 2 is needed) - with this script, the problem completely goes away for all cases.

Change History (5)

comment:1 Changed 11 months ago by Gert Döring

Owner: set to Gert Döring
Status: newaccepted

comment:2 Changed 2 months ago by Gert Döring

Summary: spurious failures with IPv6 over TAP on FreeBSD 11 and 12 (ND6_IFF_IFDISABLED)spurious failures with IPv6 over TAP on FreeBSD 12 (ND6_IFF_IFDISABLED)

Unfortunately, this is a race condition somewhere - so even if I add an explicit "ifconfig inet6 -ifdisable" to OpenVPN, FreeBSD will turn it back on behind our back.

This is from a test run, running "sleep 1 ; ifconfig tun0" in a loop...

tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        inet6 fe80::250:56ff:fe9c:dffb%tun0 prefixlen 64 tentative scopeid 0x3
        inet6 fd00:abcd:204:2::1000 prefixlen 64 tentative
        inet 10.204.2.6 --> 10.204.2.5 netmask 0xffffffff
        groups: tun
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        Opened by PID 34820
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        inet6 fe80::250:56ff:fe9c:dffb%tun0 prefixlen 64 tentative scopeid 0x3
        inet6 fd00:abcd:204:2::1000 prefixlen 64 tentative
        inet 10.204.2.6 --> 10.204.2.5 netmask 0xffffffff
        groups: tun
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        Opened by PID 34820

As with the --up script, there has to be a sleep() between "ifconfig inet6 $addr" and "ifconfig inet6 -ifdisable".

As the ticket title says "FreeBSD 11" I have re-tested this, but cannot reproduce the problem there (without the workaround, running master as of today). It never says "I tested and saw it there", so I must have been confused :-) ("man ifconfig" mentions that flag, but I've never seen it get set on its own).

Patch is trivial, if ugly...

--- a/src/openvpn/tun.c
+++ b/src/openvpn/tun.c
@@ -1038,6 +1038,23 @@ do_ifconfig_ipv6(struct tuntap *tt, const char *ifname, int tun_mtu,
     openvpn_execve_check(&argv, es, S_FATAL,
                          "generic BSD ifconfig inet6 failed");
 
+#if defined(TARGET_FREEBSD) && __FreeBSD_version >= 1200000
+    /* FreeBSD 12 grew some sort of race condition where sometimes
+     * the IPv6 side of a tun interface is stuck in "IFDISABLED"
+     * in "ifconfig tun0" output, and the assigned IPv6 address is not
+     * working ("tentative").  Workaround in here, for now.
+     *
+     * the sleep is needed because there is a race condition, so turning
+     * it off immediately can lead to "kernel turns it on again, stuck!!"
+     */
+    sleep(1);
+    argv_printf(&argv, "%s %s inet6 -ifdisabled", IFCONFIG_PATH, ifname);
+    argv_msg(M_INFO, &argv);
+
+    openvpn_execve_check(&argv, es, S_FATAL,
+                         "FreeBSD BSD 'ifconfig inet6 -ifdisabled' failed");
+#endif

comment:3 Changed 2 months ago by Gert Döring

Patch on the list, needs to go into 2.4 and 2.5

comment:5 Changed 4 weeks ago by Gert Döring

Resolution: fixed
Status: acceptedclosed

The FreeBSD folks have pointed me to /etc/pccard_ether and ipv6_activate_all_interfaces="YES" - which helps understand the cause of the problem.

Unfortunately, they see this as "works as designed" (if you want IPv6, enable it in rc.conf) while I see this from an OpenVPN user point of view, who might not have or want IPv6 "in his normal environment" but *does* want IPv6 in his VPN, without wanting to configure anything outside of OpenVPN...

Until we can find a better agreement, I am merging this. The sleep(1) looks race'y, but since the critical time window is very short, this is good enough (it's hard enough to purposely hit the problem - it needs specifics, like "disk activity" and "slow VM storage").

I am closing this one for now. If I find a better workaround, I might reopen and comment again.

commit 5e19cc2c1bf22d44f10dc585ba69d48951dc7a86 (master)
commit 6dac3342d851995bdee75db4b52cfb3a3f9232e3 (release/2.5)
Author: Gert Doering
Date: Thu Jul 23 14:19:49 2020 +0200

Workaround FreeBSD 12+ race condition on tun/tap open with IPv6.

Note: See TracTickets for help on using tickets.