Opened 9 years ago

Closed 7 years ago

#425 closed Bug / Defect (fixed)

Topology subnet doesn't work on FreeBSD 10.0

Reported by: garga Owned by: Gert Döring
Priority: major Milestone: release 2.3.14
Component: Networking Version: OpenVPN 2.3.4 (Community Ed)
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords: topology subnet


When you run openvpn client on FreeBSD 10 and connect to a server that sets topology to subnet, it doesn't work properly because tun interface is set as POINTTOPOINT when it should be changed to BROADCAST.

Attached patch fix it.

Attachments (1)

patch-src__openvpn__tun.c (785 bytes) - added by garga 9 years ago.

Download all attachments as: .zip

Change History (21)

Changed 9 years ago by garga

Attachment: patch-src__openvpn__tun.c added

comment:1 Changed 9 years ago by Gert Döring

Please provide a log explaining why it is not working today.

Pointtopoint/broadcast is meaningless for a tun interface (it is always "point to point"
from the view of the kernel, as there is no next-hop resolution involved), and the existing code works nicely on FreeBSD up to 9.2 - haven't tested 10.0 yet. The important bit is to have a route pointing the subnet in question into the tun.

To the contrary, setting the interface to BROADCAST breaks IPv6, which is why this was actually reverted about 3 years ago. So we're not going to set IFF_BROADCAST - if needed, we can add code to add a subnet route to the tun interface (in case the ifconfig is not sufficient anymore).

comment:2 Changed 9 years ago by Eric Crist

Resolution: notabug
Status: newclosed

This is working as expected. If you want broadcast traffic, you should be using the TAP adapter. I use OpenVPN extensively on FreeBSD 10 and we have experienced no problems.

comment:4 Changed 9 years ago by Gert Döring

Resolution: notabug
Status: closedreopened

I'm reopening it for now. I want to either have my own freebsd 10 buildslave up and running and test it myself, or want to see a logfile showing failure.

I have seen a failure case with "topology subnet" on an earlier FreeBSD version, but when testing, it turned out that the culprit was (in this case) conflicting routes from previous tests, so on a pristine routing table, everything worked nicely.

In the end, there's two aspects to "broadcast" here:

  • whether or not all packets that are matched by ip+netmask are sent by the kernel into the tun interface - this is independent of broadcast/pointtopoint, and purely a matter of "are the routes installed properly". We should get this right.
  • whether or not the system tries to do next-hop resolution, specifically neighbour discovery for IPv6 on the tun. Setting the tun to BROADCAST makes FreeBSD run ND for IPv6, which fails for us, because OpenVPN is lacking proper fe80:: handling (the NS packets are just dropped by the server side). This is the reason why the original code that did set IFF_BROADCAST was reverted somewhere in the beta phase before 2.3.0
  • most interesting for me to understand whether our horizon is too limited: there could be use cases where programs refuse to "see" a tun interface because said programs "only work on BROADCAST interfaces" - or maybe some sort of broadcast communication is attempted but does not work, because replication of "packets to the broadcast address" to "all clients" doesn't work. This would actually be independent of the IFF_BROADCAST setting (again) as the client openvpn will forward all packets to the server anyway, not having local "intelligence", while the openvpn server has no idea what the OS thinks it's tun interface is configured for - if it comes in via the tun FD, all information left is what is in the IP header.

So, points 1+2 can be summarized as "it should work without, if not, I need to fix the routing setup". Point 3 is "let me see the use case" :-)

comment:5 Changed 9 years ago by Gert Döring

So. Installed 10.0p7 and openvpn git master today, tested with topology subnet, announcing a /24 from the server.

This is what I have:

fbsd100.ov$ ifconfig tun0
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500

inet --> netmask 0xffffff00
inet6 fd00:abcd:194:3::100a prefixlen 64

fbsd100.ov$ netstat -rn UGS 0 148 tun0

and I can see that packets to addresses out of the /24 are properly sent into the tunnel (verified with tcpdump on the server's tun interface). So I think this is good enough for "normal usage".

I'm still curious if there is something I am not aware that needs IFF_BROADCAST. Please let me know.

comment:6 Changed 9 years ago by garga

Let me show the case that happened to me and lead me to start looking for a solution. This same scenario works fine on pfSense 2.1 (FreeBSD 8.3) but fails on pfSense 2.2 snapshots (FreeBSD 10-STABLE).

I connect to a server configured with 'topology subnet', It sends me 'ifconfig' and I end with tun0 set as you mentioned in comment #5. I also have a route for network pointing to tun0 and can reach gateway without problems.

But server also sends me 2 networks that I should have access, and, those routes are created in the system using as gw.

Here the issue happens, those routes point to lo0 and I end up with a routing loop. Here is part of my routing table: UGS lo0 UGS lo0 UGS tun0 link#14 UH tun0

Am I doing something wrong here? Let me know if you need more details

comment:7 Changed 9 years ago by Eric Crist

This really appears to be a routing issue. Your second route, overlaps the route passed in the ifconfig push.

Last edited 9 years ago by Eric Crist (previous) (diff)

comment:8 Changed 9 years ago by garga

It doesn't overlaps it, it contains it, and every OS chooses the most specific match. BTW, this same scenario works fine on FreeBSD 8.3, linux, OSX, Windows.

And on FreeBSD 10, If it's not 'topology subnet' I would end up with a tunnel like ' ->' and the /16 routes would go to tun0 instead of lo0. Same happens after apply this patch and change interface to IFF_BROADCAST and I end up with this items on routing table: UGS tun0 UGS tun0 link#14 U tun0

comment:9 Changed 9 years ago by Gert Döring

ecrist: /24 beats /16, so this is perfectly fine. (A default route would always overlap everything, and it still works).

garga: this is interesting. I'd need to see the openvpn log. The funny thing is that my test setup does the same thing, pushing an encompassing /16 for the /24, and my automated ping test verifies that this works.

My routing table with the extra route:

Destination Gateway Flags Refs Use Netif Expire UGS 0 0 tun0 UGS 0 0 tun0 link#3 UH 0 0 tun0

and I can ping (a lo0 on the openvpn server) fine.

This is what my OpenVPN log has to say about the routes:

Wed Jul 9 22:11:39 2014 PUSH: Received control message: 'PUSH_REPLY,ifconfig-ipv6 fd00:abcd:194:3::100a/64 fd00:abcd:194:3::1,route,route-ipv6 fd00:abcd:194::/48,tun-ipv6,route-gateway,topology subnet,ping 10,ping-restart 30,ifconfig'
Wed Jul 9 22:11:39 2014 /sbin/ifconfig tun0 mtu 1500 netmask up
Wed Jul 9 22:11:39 2014 /sbin/route add -net
Wed Jul 9 22:11:39 2014 /sbin/route add -net
Wed Jul 9 22:11:39 2014 Initialization Sequence Completed

(leaving off all the IPv6 stuff)

Garga, when you end openvpn, are any 172.* routes still there? OpenVPN used to do this every now and then, leave broken routes around if aborted, but it should properly clean up behind itself nowadays - but if there is anything 172.* before OpenVPN starts, it might confuse "route"

comment:10 Changed 9 years ago by garga

cron2, are you testing it on a FreeBSD 10?

PUSH: Received control message: 'PUSH_REPLY,ifconfig-ipv6 xxxx:xxx:11:f111::1000/64 xxxx:xxx:11:f111::1,route,route-ipv6 xxxx:xxx:11::/48,dhcp-option DNS,dhcp-option DNS,route,tun-ipv6,route-gateway,topology subnet,ping 10,ping-restart 60,ifconfig'

comment:11 Changed 9 years ago by Gert Döring

FreeBSD 10.0-RELEASE-p7, as I said. OpenVPN git master, but that code should be identical to 2.3.4.

Please show the bits from the log where "/sbin/route add ..." is called (might need "--verb 3"), and please also show "netstat -rn -f inet" before OpenVPN starts, and after it completed initialization.

comment:12 Changed 9 years ago by garga

Nevermind, you can close the ticket. It works as expected on stock FreeBSD, it's specific routing issue on pfSense. Thanks!

comment:13 Changed 9 years ago by Eric Crist

Resolution: invalid
Status: reopenedclosed

You're both right - I'm an idiot.

comment:14 Changed 7 years ago by Gert Döring

Resolution: invalid
Status: closedreopened

Supposedly it's broken in 11.0-RELEASE now also on "stock FreeBSD". Reopening, revisit the issue.

Seems the "myself" address is no longer logically tied to the tun0 interface but to lo0, so the old trick of pointing a route to "my own address" to mean "go to *this* interface" no longer works.

For tun interfaces, just using -ifp tun0 should be reasonable *if* it's a route intended to go to the tun if (--route has somewhat funny semantics for IPv4) and that would be a fairly simple fix. Not in the next 10 days, though, too busy.

comment:15 Changed 7 years ago by Gert Döring

Owner: set to Gert Döring
Status: reopenedaccepted

comment:16 Changed 7 years ago by Gert Döring

Milestone: release 2.3.14

nicely pinpoints it - it was an change in FreeBSD which broke it, but arguably our approach wasn't "really nice" before that, it just happened to work.

I'll tackle it, should make 2.3.14

comment:17 Changed 7 years ago by Gert Döring

this somewhat overlaps with #481 as well

comment:18 Changed 7 years ago by Gert Döring

patch is on the list, works for me on FreeBSD 11.0-RELEASE-p3 and 7.4-RELEASE ("most recent and oldest system I have"), both client and server.

Feedback welcome.

Date: Tue, 8 Nov 2016 13:45:06 +0100
From: Gert Doering <gert@…>
To: openvpn-devel@…
Subject: [Openvpn-devel] [PATCH] Repair topology subnet on FreeBSD 11

comment:19 Changed 7 years ago by garga

Great! I've imported the fix to pfSense's ports tree [1] and users reported it fixed the issue [2]. Thanks!


comment:20 Changed 7 years ago by Gert Döring

Resolution: fixed
Status: acceptedclosed

garga, thanks for testing and reporting back - this sped up our review process quite a bit :-)

Here we go...

commit a433b3813d8c38b491d2baa7b433973f2d6cd7c6 (master)
commit 446ef5bda4cdc75d4cb955e274846faff0181fd3 (release/2.3)
Author: Gert Doering
Date: Tue Nov 8 13:45:06 2016 +0100

Repair topology subnet on FreeBSD 11

in-tree, and will show up in 2.3.14 (whenever that's going to hit the shelves)

I'm closing this ticket as there is nothing more to do - if it turns out something is still broken, feel free to reopen (or create a new ticket and point to this one for context).

Note: See TracTickets for help on using tickets.