Opened 13 years ago

Closed 8 years ago

Last modified 8 years ago

#71 closed Bug / Defect (fixed)

Windows 7 (and Vista) - tunnel fails after resume from Sleep/Standby

Reported by: billb3 Owned by: Samuli Seppänen
Priority: major Milestone:
Component: Generic / unclassified Version: OpenVPN 2.3-beta / 2.3-RC (Community Ed)
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords: windows 7 win7 vista wake resume sleep standby hibernate
Cc:

Description

I have run into an issue when using OpenVPN (last tested with 2.1.4 & 2.2b5) on Windows 7. I install OpenVPN without errors and configure the service to run automatically. The tunnel starts and works normally when started manually or when the computer is started. The issue occurs if I put the laptop into sleep/standby mode. When I "wake" the computer, the tunnel is down. No traffic is passed, there is nothing in the openvpn log file. Once I restart the service, the tunnel works normally again. On Windows XP (as expected), the tunnel is reestablished when the computer wakes.

I have attached a few files:

"win7-openvpn.log" includes the results of the following operation. I am running Windows 7 Ultimate with the latest updates.

  • I manually started the OpenVPN service at 11:44:00.
  • I tested the tunnel and it was passing traffic normally.
  • I put the machine in sleep mode at approx 11:44:20
  • I "woke" the machine right at 11:45.
  • The tunnel was not able to pass any traffic. I let it run for approx 10 minutes (log only goes for 3 min, but contents of log were the same from minute 3-10)

In addition, I have included the results of "route print" before and after "sleep". ("routes before sleep.txt" & "routes after wake.txt")

Please let me know what I can do to help test/fix this bug!

This was originally reported on the mailing list here: http://sourceforge.net/mailarchive/forum.php?thread_name=17110127472.20100916143418%40gmxpro.de&forum_name=openvpn-users

I was directed to another bug, but that appears to be slightly different: https://community.openvpn.net/openvpn/ticket/56

Attachments (5)

routes after wake.txt (3.6 KB) - added by billb3 13 years ago.
routes before sleep.txt (4.2 KB) - added by billb3 13 years ago.
win7-openvpn.zip (16.2 KB) - added by billb3 13 years ago.
INFRA-VPN.log (50.5 KB) - added by Coren42 8 years ago.
Reconnect Problem
resume-no-vpn-connectivity.log (27.4 KB) - added by Simon Deziel 8 years ago.
no connectivity after resume

Download all attachments as: .zip

Change History (55)

Changed 13 years ago by billb3

Attachment: routes after wake.txt added

Changed 13 years ago by billb3

Attachment: routes before sleep.txt added

Changed 13 years ago by billb3

Attachment: win7-openvpn.zip added

comment:1 Changed 13 years ago by roentgen

A comment to get notifications about this bug.

comment:2 Changed 13 years ago by sys

Confirmed here on Win7 Professional x64.

comment:3 Changed 13 years ago by adapted.cat

I had the same issue on Win 7 Home Premium x64. Here's my workaround:

Create two new tasks in task scheduler, which run from a user account with elevated privileges, whether logged in or not, without storing the password. Each of these tasks triggers on event System:Power-Troubleshooter:1 (this is wake up from sleep).

The first task has no delay and no conditions, and runs "C:\Windows\System32\sc.exe" with arguments "stop OpenVPNService" (note the lack of spaces in the service name).

The second task is delayed for 10 seconds or so with no conditions (though I recommend killing it if it lasts more than a minute and retrying if it fails), and runs "C:\Windows\System32\sc.exe" with arguments "start OpenVPNService"

You could of course have a single event to run a batch script, but that introduces some security concerns so running sc directly is probably the way to go. You need the two steps because neither sc nor net have a "restart" command - just "start" and "stop."

Anyway, this is far from ideal, and would be better solved by OpenVPN itself, but it does fix the immediate problem. If you have trouble getting this to work, you may need to play around at the command line (in a shell with administrator privileges) to get the command arguments right.

comment:4 Changed 13 years ago by utkuerd

I'm having this issue for a couple of years with Vista and 7, still exists in latest 2.2.0 version on Windows 7 Home Premium x64. By the way this problem does not exist if you use OpenVPN GUI instead of service. In that case sleep or hibernate does not disturb the stability of VPN. However you need to start OpenVPN GUI at start up with elevated previleges and this might be a little tricky.

comment:5 Changed 13 years ago by Tenerezza

I can confirm that this does either not work in Windows 7. As for me it's even worser then that, after put the laptop back from sleep mode ( or hybernate mode ) same effect for both, One Core of my laptop is being used up by the OpenVPN service. I have 4 but it still eat off the battery really fast. I'm not exactly sure what the service is doing at that point but it's clear it's not coded to handle sleep or hybernate.

For an laptop user with this I see this as an really important bug to fix.

comment:6 Changed 12 years ago by Daniel S

Any update on this bug? I'm having the exact same problem as described above.

comment:7 Changed 12 years ago by fkurth

Is there any progress with this bug? Having this here on multiple Win 7 machines. What can i do to support fixing?

comment:8 Changed 12 years ago by ert

Could it be related to http://community.openvpn.net/openvpn/ticket/97 ?
I think I found the error, could someone try my proposed fix please?

comment:9 Changed 12 years ago by utkuerd

2.3alpha3 build claims fixing bug #97. However, still routes are gone after sleep/hibernate. It means two bugs are unrelated or they are related but the bug #97 is still there.

comment:10 Changed 12 years ago by billb3

Just want to confirm that 2.3alpha3 appears to fix this issue for me on Win7 x64.

comment:11 Changed 11 years ago by Samuli Seppänen

Can others confirm that this issue is gone in 2.3_alpha3 and later?

comment:12 Changed 11 years ago by Samuli Seppänen

Keywords: hibernate added

comment:13 Changed 11 years ago by Samuli Seppänen

Version: 2.1.2 / 2.1.32.3-beta / 2.3-RC

Let me rephrase my previous comment... can someone reproduce this on the latest openvpn release?

comment:14 Changed 10 years ago by hamzen

Yes :(
Sadly the error is still there, in 2.3.0, and even in 2.3.2
It happens only at Win7_64bit and Win8_64bit systems.

Tried EVERYTHING in the last 2 years, nothing helps.

comment:15 Changed 10 years ago by billb3

OP, here! This bug has been fixed for a while for me, at least since 2.3. I am using Win7 64-bit for all of my systems without issues.

comment:16 Changed 10 years ago by KarlBauman

Windows 8.1 64bit with the same problem. Client Daemon hangs up and uses 32% of my I5 CPU.
Happens either after sleep or reconnecting to Wireless networks.

Tried with this one http://swupdate.openvpn.org/community/releases/openvpn-install-2.3.2-I003-x86_64.exe and also this one http://swupdate.openvpn.net/downloads/openvpn-client.msi
Both eat 32%.

Also discovered that with second one outgoing traffic is growing like crazy - 500Mb/minute, but task manager doesn't display any outgoing traffic, so probably there is some infinite outgoing traffic loop. This happened in few minutes after connecting to VPN server https://www.dropbox.com/s/3pha9tkkq6aht6q/Screenshot%202014-03-26%2001.41.35.png

When Disconnected, CPU goes back to 0%. Since both clients act the same, I think that it could be because of Tap network interface driver not being compatible with 64Bits.

Developers are welcome to use my PC remotely for testing, just let me know.

Last edited 10 years ago by KarlBauman (previous) (diff)

comment:17 Changed 10 years ago by maenvam

I'm having the same issues even since I upgraded to 2.3.3-I002 because of the ssl issue. I'm on win7 x64.

The symptoms are the same: after waking up from sleep, the openvpn log says that everything is ok, but my ip is still my original ip, not the server's I'm connecting to. So I have to manually restart the service (actually, restart causes an error 32, permission error. I have to stop the service, wait a few seconds and start it).

When comparing the route print result, there are differences between before and after sleep.

comment:18 Changed 9 years ago by Simon Deziel

I can confirm the bug still exists on Windows 7 (32bit) with OpenVPN 2.3.4-I604-i686. The VPN routed ranges are gone after resuming from sleep. The VPN connection remains active somehow though.

This also seems to trigger a nasty behaviour on the client side where it presumably goes into an error loop that never ends. When "--verb" != 0, this will fill the HDD at an alarming rate. I had to set "--verb" to 0 because the log file would take up all the free space (80GB+).

comment:19 Changed 9 years ago by atze

OpenVPN 2.3.5 on Win7 32bit shows the following log directly after waking up the system:

Sat Dec 06 13:31:44 2014 TUN/TAP I/O operation aborted, exiting
Sat Dec 06 13:31:44 2014 Exiting due to fatal error
Sat Dec 06 13:31:44 2014 C:\Windows\system32\route.exe DELETE 10.x. ....
Sat Dec 06 13:31:44 2014 Warning: route gateway is not reachable on any active network adapters: 10.x.x.x
Sat Dec 06 13:31:44 2014 Route deletion via IPAPI failed [adaptive]
Sat Dec 06 13:31:44 2014 Route deletion fallback to route.exe
Sat Dec 06 13:31:44 2014 env_block: add PATH=C:\Windows\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
[.... repetition of route messages for each route ...]
Sat Dec 06 13:31:45 2014 Closing TUN/TAP interface

Here #110 jumps in, if the service would just be restarted (or "correctly" die), as it is expected from a windows service, the problem could be solved, but currently in my installation the openvpnserv.exe shim is the remaining problem because it does neither restart openvpn nor terminate itself.

comment:20 Changed 9 years ago by Samuli Seppänen

So if I understood this correctly this bug could be fixed by having openvpnserv.exe react to suspend events (like this?) by shutting itself down. When the computer wakes up, Windows service control will then restart it.

comment:21 Changed 9 years ago by B_Reiter

This is still happening with 2.3.6 (OpenVPN 2.3.6 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Dec 1 2014) on Windows 7.

At every single sleep/resume cycle, the following is printed in the log and the connection stops.

Tue Dec 09 14:27:21 2014 Initialization Sequence Completed
Tue Dec 09 14:54:03 2014 TUN/TAP I/O operation aborted, exiting
Tue Dec 09 14:54:03 2014 Exiting due to fatal error
Tue Dec 09 14:54:03 2014 Warning: route gateway is not reachable on any active network adapters: 192.168.11.5

The OpenVPN-service keeps running, unfortunately.

Is there any configuration setting that fixes this?
Currently in use is this:

remote ------------ 1195
dev tun
proto udp
tls-client
tls-auth ta.key 1
ca ca.crt
cert client.crt
key client.key
persist-key
comp-lzo yes
nobind
pull
resolv-retry infinite
ns-cert-type server
cipher AES-192-CBC
dhcp-option DNS 192.168.16.11
dhcp-option DNS 192.168.16.15
dhcp-option DOMAIN ---------.local
route-metric 1500

comment:22 Changed 9 years ago by B_Reiter

I have done some testing. For my setup, version 2.3.6 is the first one to cause problems. The issue also does not seem to be the TAP-driver.

I've tested the following OpenVPN/Tap-Driver combinations:
OVPN 2.3.6 + TAP 9.21.1: Crash
OVPN 2.3.6 + TAP 9.21.0: Crash
OVPN 2.3.4 + TAP 9.21.0: No Crash, Reconnect OK
OVPN 2.3.4 + TAP 9.21.1: No Crash, Reconnect OK

Based on the time stamps it also seems that the log entry

TUN/TAP I/O operation aborted, exiting

is written during suspend, not after or during resume.

Can we expect an updated OpenVPN Installer soon, or should we downgrade to 2.3.4 until the issue is solved?

comment:23 Changed 9 years ago by jeffreyt

Has anyone else tried this?

https://openvpnwinsvc.codeplex.com/

I've been struggling with this problem for a while as well and this app seems to do the trick.

It purports to take over the service part and run openvpn.exe under it. Getting it running was a little tricky. I had to modify my .ovpn config file and create another OpenVPN directory in the x86 program files. I can go into more detail if people want.

Sorry if this is not the right place to post this but after reading other people's comments, I wasn't sure if other people were aware of it or not. The source code is posted so maybe some smart people can figure something out.

comment:24 Changed 9 years ago by Samuli Seppänen

The openvpnwinsvc looks interesting. One other alternative is NSSM that is mentioned in the new FAQ item on this issue.

I'll look into the alternatives to openvpnserv.exe and see if one of them could be integrated into the official builds.

comment:25 Changed 9 years ago by Dwig

I'm using OpenVPN as a client, running under OpenVPN GUI, rather than as a service; this is because the server requires credentials to create the connection (I'm running under Win 7, using the 64-bit version). As with others here, I often, but not always, find that I can't reconnect after a Hibernate (the GUI can't make a new connection). In this case, I find that there's an openvpn.exe process; killing it allows me to make the connection as usual.

I played with NSSM, but found out that doesn't work when credentials are required.

Here's a possible workaround, for those willing and able to modify the GUI: when a timeout happens and there's an instance of openvpn.exe running, kill it and restart.

comment:26 Changed 9 years ago by Samuli Seppänen

We decided to take a shot at integrating NSSM into OpenVPN 2.4. It will not help solve this problem for people using a GUI, but it should make OpenVPN running as a Windows Service much more robust. NSSM will be optional obviously.

comment:27 in reply to:  26 ; Changed 9 years ago by ssamppa

Replying to samuli:

We decided to take a shot at integrating NSSM into OpenVPN 2.4. It will not help solve this problem for people using a GUI, but it should make OpenVPN running as a Windows Service much more robust. NSSM will be optional obviously.

Any news on this? I would be happy to help testing if you have already a test build.

comment:28 in reply to:  27 Changed 9 years ago by Samuli Seppänen

Replying to ssamppa:

Replying to samuli:

We decided to take a shot at integrating NSSM into OpenVPN 2.4. It will not help solve this problem for people using a GUI, but it should make OpenVPN running as a Windows Service much more robust. NSSM will be optional obviously.

Any news on this? I would be happy to help testing if you have already a test build.

There are no test builds yet, but I did test manually integrating NSSM and OpenVPN and it "seemed to work great". You can definitely help by testing nssm + openvpn according to the Wiki instructions I wrote earlier. If you encounter any issues with NSSM please let me know.

There is also one other thing I'd need help with. The nssm.exe's configuration frontend is too complex for our use case, so we'd need a simplified GUI. It looks like a simple local HTML application (.hta) would do the trick without excessive amount of work. I'd like to avoid having to write VBScript, but to be honest, that seems much easier in this case than JScript (~Javascript). Anyways, I already have some very crude HTML+JScript code that does something, but which is not nearly finished. If somebody else wants to participate in this project I will gladly publish the horrible mess of a code I now have on GitHub.

comment:29 Changed 8 years ago by Rozi

I had the same problem when upgrading from 2.3.0 to 2.3.8.

Thank you for pointing me to NSSM. Not only did it solve the openvpn.exe death on sleep problem; it handles each OpenVPN connection as a separate service. I like that feature a lot:

  1. You can control each connection individually (like in OpenVPN for Linux),
  2. You can set additional service dependecies on per-connection basis (i.e. stunnel)...

comment:30 Changed 8 years ago by arrmo

Hi,

Seeing the same thing here in 2.3.8, but one more symptom - when I resume, often if I restart OpenVPN and connect to the server, I am unable to get a (client) IP address. To get it working again, I have to kill OpenVPN, then disable the TAP adapter, re-enable it, and finally restart OpenVPN. Then it will work. So it's like the TAP adapter has to be reset for some reason.

Is anyone else seeing this?

Thanks!

comment:31 Changed 8 years ago by Gert Döring

Owner: set to Samuli Seppänen
Status: newassigned

2.3.9 has a bug fix for the openvpn abort - not perfect (as we're still not on NSSM) but this should fix the generic case

commit 0d4ba251879c702b9474e26ff73a4f559d922d4f
Author: Selva Nair <selva.nair@…>
Date: Wed Nov 4 13:59:38 2015 -0500

Fix termination when windows suspends/sleeps


When TUN/TAP I/O operation is aborted, restart with a SIGHUP instead of
terminate. The abort error from TAP is often triggered by system suspend
which is fully recoverable on resume. Catastrophic events will get caught
later during the restart. This solves the abnormal termination during
suspend/resume.


Signed-off-by: Selva Nair <selva.nair@…>
Acked-by: Gert Doering <gert@…>
Message-Id: <1446663578-14471-1-git-send-email-selva.nair@…>
URL: http://article.gmane.org/gmane.network.openvpn.devel/10438

Assigning the ticket to Samuli now - either NSSM or close :-)

comment:32 Changed 8 years ago by bvpn

2.3.9 64bit running in server mode still has an issue for Windows wake from suspend/sleep.

"All TAP-Windows adapters on this system are currently in use.
Exiting due to fatal error"

The issue does not happen consistently (sometimes in rare cases during testing openvpn 2.3.9 resumes correctly).

comment:33 Changed 8 years ago by Gert Döring

@bvpn: can I have a log file how that abort looks like, with --verb 3 or --verb4, please?

Is this with the ndis5 or ndis6 tap driver (I00x or I60x installers)?

Run from GUI or run from Service?

comment:34 Changed 8 years ago by Coren42

@cron2: I have a similar problem.

With the current stable client Windows 64 bits client, v 2.3.10 I602, it fails to reconnect after 1 or 2 minutes. If I manage to be quick enough to re-activate TAP Driver, my reconnection is a success. Otherwise it's a failure.

I have attached a log with verb 4 on this ticket. You can see on it a successful connection at 15:02:44 (https://community.openvpn.net/openvpn/attachment/ticket/71/INFRA-VPN.log#L362)

And when it tries to reconnect, I have a "TUN/TAP I/O operation aborted, restarting" which does not manage to restart properly. The client successfully desactivate the TAP Adapter, but is unable to re-activate it.

Can you do something about this ? I can provide more information if needed.

Last edited 8 years ago by Coren42 (previous) (diff)

Changed 8 years ago by Coren42

Attachment: INFRA-VPN.log added

Reconnect Problem

comment:35 Changed 8 years ago by Simon Deziel

I'm running the client as a service (OpenVPN 2.3.10 I602 32bit) and I also noticed that after suspend/resume, the connectivity isn't re-established. If logs are needed let me know, thanks.

comment:36 in reply to:  34 Changed 8 years ago by Selva Nair

Replying to Coren42:

With the current stable client Windows 64 bits client, v 2.3.10 I602, it fails to reconnect after 1 or 2 minutes. If I manage to be quick enough to re-activate TAP Driver, my reconnection is a success. Otherwise it's a failure.

I have attached a log with verb 4 on this ticket. You can see on it a successful connection at 15:02:44 (https://community.openvpn.net/openvpn/attachment/ticket/71/INFRA-VPN.log#L362)

And when it tries to reconnect, I have a "TUN/TAP I/O operation aborted, restarting" which does not manage to restart properly. The client successfully desactivate the TAP Adapter, but is unable to re-activate it.

The change from exit to restart in openvpn on TAP errors was based on the observation that TAP driver (ndis6) suspends on power events but does come back up on resume. Your logs seems to show otherwise.

Is this win7? Started from GUI or service? I haven't seen such errors, but win7 + tap6 does sometimes behave strangely so there could be a bug somewhere.. When this happens, can you reconnect if you try again after a while _without_ reactivating the TAP interface?

Also could you please check for any network related errors in the event log?

If this is pre win8, an option is to revert back to the ndis5 driver.

comment:37 in reply to:  35 ; Changed 8 years ago by Selva Nair

Replying to simon.deziel:

I'm running the client as a service (OpenVPN 2.3.10 I602 32bit) and I also noticed that after suspend/resume, the connectivity isn't re-established. If logs are needed let me know, thanks.

Yes, logs please.

comment:38 Changed 8 years ago by Coren42

@selvanair: You're right, this is a different problem. Eventually, I've managed to find the culprit: A specific & custom-made service named "e-buro". I no longer encounter the problem when I stop it.

Changed 8 years ago by Simon Deziel

no connectivity after resume

comment:39 in reply to:  37 Changed 8 years ago by Simon Deziel

Replying to selvanair:

Replying to simon.deziel:

I'm running the client as a service (OpenVPN 2.3.10 I602 32bit) and I also noticed that after suspend/resume, the connectivity isn't re-established. If logs are needed let me know, thanks.

Yes, logs please.

Please see the resume-no-vpn-connectivity.log​ attached. Thanks

Edit: the attached log is from a 64 bit client but I can reproduce with 32 bit as well.

Last edited 8 years ago by Simon Deziel (previous) (diff)

comment:40 Changed 8 years ago by Selva Nair

Thanks for the log. The disconnect happens due to route deletion error.
When TAP I/O fails at suspend/sleep, and the connection restarts, its normal for route deletion to fail as the interface is no more there. In case of ipv4, route addition/deletion errors are not fatal. But, netsh route deletion failure is treated as FATAL. I had no idea this is the case.

The fix we added in commit 0d4ba25187.. will not work in this case unless that error is made not-fatal.

@cron2: Can ipv6 route deletion errors be made NON_FATAL?

comment:41 Changed 8 years ago by Gert Döring

@selvanair: right you are. Never occured to me that route add/route del are not necessarily not a symmetric issue ("what is added must be removable") because system events could be interfering here.

*Normally*, an error on route DEL would not be overly harmful either (because we're exiting anyway) - but in this case where OpenVPN needs to do a full tun restart, it's a bad idea. So yes, let's change this - I'll do a patch.

comment:42 Changed 8 years ago by Selva Nair

FWIW, looking at this again, its not route deletion but address deletion that finally throws the fatal error. I wrongly thought it was the route error that caused the exit.

comment:43 Changed 8 years ago by leifnel

Just to keep your interest awake: The problem seem to exist on windows 10 pro 64bit too.

comment:44 Changed 8 years ago by Samuli Seppänen

Many of these suspend/resume problems will be solved by openvpnserv2. In my tests on a Windows 7 pro laptop + openvpnserv2 OpenVPN reconnected consistently after resuming. Now that 2.4-alpha1 is only a few small bits away I will need to start integrating openvpnserv2 into the installer.

The latest OpenVPN-GUI versions have suspend/resume-related changes, which might alleviate/fix this issue. Anyone experiencing these issues should try out the Git "master" snapshots which include the latest OpenVPN-GUI:

Use the latest "openvpn-install-master" snapshot you can find, because the older ones don't have updated OpenVPN-GUI. Right now the latest ones are these:

Please let us know if the snapshot installers fix the issue for you, or if they don't.

comment:45 in reply to:  43 Changed 8 years ago by Selva Nair

Replying to leifnel:

Just to keep your interest awake: The problem seem to exist on windows 10 pro 64bit too.

A fix has been merged in today (May 16). It would great if you could test it. 64bit snapshot builds including this for release(2.3) and master are at:

comment:46 Changed 8 years ago by Samuli Seppänen

This installer include fully integrated openvpnserv2.exe, which should handle suspend/resume gracefully. The new openvpnserv2-based service is called "OpenVPN Service", whereas the old (buggy) is called "OpenVPN legacy service". You should not have both enabled at the same time, or things will break horribly. You can enable the "OpenVPN Service" using the services manager, or with various command-line tools (sc.exe, net.exe, Set-Service Powershell CmdLet).

The installer I linked to also contains properly integrated Interactive Service and OpenVPN-GUI that supports it, so you don't need to run OpenVPN-GUI as admin anymore.

Afaik both OpenVPN-GUI + Interactive Service and openvpnserv2.exe should handle suspend and resume correctly. Please test the above installer and let us know if it the problems you've been encountering are fixed.

comment:47 Changed 8 years ago by atze

The linked installer resolves the issue in my use case(*) completeley - great!

(*) Win7 x64, tunnels configured as system service do not restart after hibernate/resume cycle

comment:48 Changed 8 years ago by Samuli Seppänen

Can someone reproduce suspend/resume issues with either openvpnserv2.exe, or with OpenVPN-GUI+Interactive Service? If not, we can close this ticket after all the required changes have been merged to openvpn and openvpn-build.

comment:49 Changed 8 years ago by Samuli Seppänen

Resolution: fixed
Status: assignedclosed

All the required changes to openvpn and openvpn-build have been merged. Apparently nobody has been able to reproduce this issue using openvpnserv2.exe or OpenVPN-GUI + Interactive Service, so I'll close this as fixed.

comment:50 Changed 8 years ago by hank

Just tried this on Windows 10 x64 with the I907 installer suggested above. Works like a charm!

Note: See TracTickets for help on using tickets.