Opened 4 years ago

Last modified 3 years ago

#595 assigned Bug / Defect

Failed to auto startup in Win10

Reported by: maverick74 Owned by: Samuli Seppänen
Priority: major Milestone:
Component: Generic / unclassified Version: OpenVPN 2.2.2 (Community Ed)
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords: openvpnserv windows
Cc:

Description

After upgrade from Win 7 to Win 10, OpenVPN fails to do his job.

Configurations in windows 7:

Installed version was OpenVPN 2.2.2.
It was configured in "Control Panel > Administrative tools > Services" to Start Automatically on Windows Startup

After migration to windows 10, the software version and configurations remained the same, but the connection is no longer established. However, in either "Services" (in control panel) and in "Services" (in the Task Manager) the system reports it as "in execution" and after a "Restart" command in "Services" the connections is established correctly

Latest version does not fix the problem.

Change History (33)

comment:1 Changed 4 years ago by Gert Döring

Owner: set to Samuli Seppänen
Status: newassigned

Yet another windows service fail :-( - we need a category "Windows Service" I think...

comment:2 Changed 4 years ago by maverick74

Hi

This is a very big pain in the *, as we now have to do this same steps (restart the service) to near 20 computers every single day.

Is there any known workaround?

Tks

Last edited 4 years ago by maverick74 (previous) (diff)

comment:3 Changed 4 years ago by Gert Döring

you could try setting start to "delayed" (if win10 still has it, was introduced with win7)...

Is there anything in the event log that would explain why the service isn't starting?

Alternatively, you could try using NSSM instead of openvpn-service - no idea whether it works on win10, but on win8 it is more robust regarding restarting openvpn on suspend/resume, so we already recommend it there...

comment:4 Changed 4 years ago by maverick74

"Delayed" didn't work. I tried it as well as a few options in "Recovery" tab.

The event log doesn't show anything important either (not event log from windows or from openvpn).

I will try NSSM - the Non-Sucking Service Manager, i suppose - to see if i understand how it works and if it solves it.

Any idea of an ETA for a bugfix?

Thanks :)

comment:5 Changed 4 years ago by maverick74

In fact i might have miss looked the openvpn log and i'm getting this errors that might help you (this is after i manually start the openvpn GUI that also fails):

WARNING: No server certificate verification method has been enabled.

ROUTE: route addition failed using CreateIpForwardEntry?: Acess denied. [status=5 if_index=4]
env_block: add PATH=C:\Windows\System32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
ERROR: Windows route add command failed [adaptive]: returned error code 1

comment:6 Changed 4 years ago by Gert Döring

Well, the route addition failure hints at "not running with enough privileges", but there's a few --ip-win32 settings that migh thelp - if CreateIpForwardEntry?() doesn't work, maybe "-ip-win32 netsh" works better.

As for an ETA for the service - well, as soon as a volunteer shows up that a) has win10 and b) enough free time to figure out what is wrong and fix it.

comment:7 Changed 4 years ago by maverick74

Hi,

your first paragraph is a little too technical for my knowledge...

thanks for your reply (it's good to have replies :)

comment:8 Changed 4 years ago by Samuli Seppänen

Tests using NSSM on Windows 10 would be most welcome, as I've only tested Windows 8.1 so far. OpenVPN-specific NSSM instructions are here.

The fix and a long-term solution is explained in the Integrate NSSM into OpenVPN ticket. We're still months rather than weeks away for really fixing this unless we get help from our community. My previous request for comments and help was mostly ignored, so I would not hold my breath waiting for the fix.

The problem is that we don't really have any Windows developers, let alone C# + WPF developers. I will take the brunt and bootstrap the NSSM configuration GUI, but as I've never coded C# nor WPF, it will take lots of time and effort. An experienced Windows GUI developer would probably whip something up in a few hours.

comment:9 Changed 4 years ago by Samuli Seppänen

Keywords: openvpnserv windows added

comment:10 Changed 4 years ago by Samuli Seppänen

Oh, and you should definitely use the latest OpenVPN 2.3.x release instead of the obsolete 2.2.2 release. We have not updated the 2.2.x branch in a very long time and we can't give any guarantees it will work properly on recent Windows versions. Moreover, the official OpenVPN 2.2.2 installers contain obsolete and horribly insecure versions of OpenSSL.

That said, openvpnserv.exe has not changed in many years, so Windows service management for OpenVPN will still suck.

comment:11 Changed 4 years ago by maverick74

Hi,

Thanks for your reply.

I've extracted nssm.exe into C:\Program Files\OpenVPN\bin as instructed, but i have two questions:

1.I created the batch file as instructed (named it Monitor.bat) but i don't know in which directory to put it. Which one is the right one?

2.Also, will this automatically start up with windows???

Thanks for your help

comment:12 Changed 4 years ago by maverick74

Hi, I've figured out part of the problem, that might help.

If we go into the task manager (i'm on Win10) and we get to the "Details" Tab only openvpnserv.exe is loaded. Openvpn.exe is nowhere to be found. Only when i force OpenVPN service to restart, openvpn.exe is loaded. Once openvpn.exe starts running the problem gets solved...

Edit: I went back from the latest release into openvpn 2.2.2 and, whit this version, i always get openvpn.exe running on startup, however, not always it will work. If i shutdown the computer and then startup, everything goes ok.

If i restart the computer it stops working. Even if after the reboot i shut it down, and start it up i can't get it working again. Only after going to TaskManager? > servives > openvpn and restart the service i can get it "fully" working again...

Last edited 4 years ago by maverick74 (previous) (diff)

comment:13 Changed 4 years ago by Eagle_Erwin

I experience the same issue with OpenVPN 2.3.8. In my setup, I have 4 different OpenVPN config files, which all have a 'management 127.0.0.1 <PORT>' option configured. At boot, most of the times three out of four configurations are actually started. This is verified by checking if the management port is in use. Sometimes (let's say: one out of three boots), all four configurations are started. In the case that only three configurations are started, always the same configuration is not started. I don't know how OpenVPN decides the order of each config to start, so I can't say if it is the first or last config that is failing.

I also noticed in the task manager that there is one openvpn.exe process started for each configuration, but not for the failing one. Restarting the service always fixes the problem.

I have not (yet) tested NSSM.

comment:14 Changed 4 years ago by jtryon

Seeing the same issue. Bump.

comment:15 Changed 4 years ago by xkjyeah

If I didn't misunderstand the issue, is the problem that something is causing OpenVPN.exe to crash/fail to start up?

And NSSM works around that by restarting the child process if it dies?

I intend to rewrite openvpnserv.exe to incorporate the auto-restart functionality. Shouldn't be too hard. So let me know if I misunderstood the problem.

comment:16 Changed 4 years ago by Eagle_Erwin

I can confirm that under certain circumstances, the openvpn.exe process fails to start after it should be spawned by openvpnsrv.exe. So as far as I can see, you did not misunderstand the problem.

I noticed a while ago that the problem only occurs if you have an active VPN connection and you shut down your Windows 10 machine and start it up again using the fast startup feature (which is enabled by default). It does not occur after a clean reboot (either manually or automatic after Windows update installation) and it does also not occur if you first disconnect the VPN connection, before shutting down the machine.

In the logs, I always noticed the following lines in the case that openvpn.exe does not start:

Mon Jan 18 18:36:32 2016 OpenVPN 2.3.9 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Dec 16 2015
Mon Jan 18 18:36:32 2016 library versions: OpenSSL 1.0.1q 3 Dec 2015, LZO 2.09
Mon Jan 18 18:36:32 2016 MANAGEMENT: TCP Socket listening on [AF_INET]127.0.0.1:1195
Mon Jan 18 18:36:32 2016 Need hold release from management interface, waiting...

I use port 1195 as the management interface for the connection that was active before I shut down my machine.

So for me, it seems that the management port is not released properly in case of a fast startup sequence, so the 'new' openvpn.exe is unable to open it after startup.

comment:17 Changed 4 years ago by xkjyeah

@Eagle_Erwin

My observations are slightly different. On Windows 10 I notice that openvpnserv.exe does not even begin to overwrite my log file. Could this be due to Windows 10's "Fast Startup" feature being actually a "resume from suspend"?

In any case, I've written a small service intended as a drop-in replacement for openvpnserv.exe.
https://github.com/xkjyeah/openvpnserv2/tree/master/bin/Release.

Because this binary (or the more fully-featured https://openvpnwinsvc.codeplex.com/) can be built entirely using the Mono toolchain, it shouldn't be difficult to include it in the build process.

Last edited 4 years ago by xkjyeah (previous) (diff)

comment:18 Changed 4 years ago by Samuli Seppänen

Could you test xkjyeah's binary? I will try to do testing myself the upcoming week. After that I can integrate it into test installers for wider testing. I'd actually prefer a drop-in replacement for ovpnserv.exe over NSSM + wrapper (=ovpnsvcsetup).

comment:19 Changed 4 years ago by Samuli Seppänen

I built my own version of xkjyeah's C# of openvpnserv.exe and placed it to C:\Program Files\OpenVPN\openvpnserv.exe on a Windows 7 64-bit laptop. Based on very brief testing the new service wrapper seems to work fine. I also tested putting the laptop to suspend. On resume it took 10-20 seconds before the VPN connection was usable again, but after that it worked great!

Please test the binary on other Windows architectures and platforms. Also, I did not do any tests with multiple simultaneously active VPN tunnels.

comment:20 Changed 4 years ago by maverick74

Hi.

Next Monday (if possible) I'll test it under our environments here as well.

I'll post a Feedback ASAP.

Thanks

comment:21 Changed 4 years ago by xkjyeah

Thanks for testing. Meanwhile to help debugging I thought I should mention a few things:

  1. Some (but not all) messages are logged to the event viewer, including but not limited to startup and shutdown messages.
  1. The service watches for process termination, and restarts a terminated process after 10seconds. Thus you could execute taskkill /f /im:OpenVPN.exe and observe the service process restart it. A message should also be logged to the event viewer when that happens. 10 seconds was arbitrarily chosen, because it felt like a good compromise between a long delay and an endlessly crashing and restarting processconsuming resources.

comment:22 Changed 4 years ago by maverick74

Hi,

I've tested it on a 6-win10-computer sample.

From this sample, 3 64bit machine went OK. After multiple restart openvpn always (100%) started OK on windows logon. From the other 3 computer (32bit architecture) only one achived the same score. On the other 2 i was not able to make it work.
I've made the same exact procedures as on the others before (removed any old version and installed the most recent one) but no luck!!!

Any ideia on what might be the problem?

Tks

comment:23 Changed 4 years ago by xkjyeah

@maverick74
I wish I had a 32-bit machine to debug the issue! Nevertheless, let's try:

  1. No offence meant, but if you have set the service to start "Automatically" without "Delayed Start", just make sure it's set to "Automatic" again.
  1. Can you check for messages in the Event Viewer? I would like to establish a couple of things: (N.B. look for OpenVpnService?-related messages under "Windows Logs\Application"

a) Whether exception messages have been written to the Event Viewer.

b) Whether the service was started successfully and/or openvpnserv.exe was running after startup. For this you could check the Task Manager and/or the Event Viewer.

c) Whether a log file was written to. For every config file <filename>.ovpn you have in \OpenVpn?\config\ you should have a corresponding <filename>.log file in \OpenVpn?\log\. Check the last modified time of the log file. Is it after startup, or before startup?

d) Does the service run if you restart it manually?

e) Any other details you might think is significant

Thanks!

Last edited 4 years ago by xkjyeah (previous) (diff)

comment:24 Changed 4 years ago by maverick74

Hi

Thanks for your fast reply :). I'm not on location now (those 2 were tested on another division.) I'll check this ASAP...

But i can answer a few right away:

1) and b): i did checked it a always after a reboot. The service was always set to start "Automatically" and the task manager always said it was in did "running", however, dispite what was told on the task manager, i was not able to establish connection.

d)It does work fine if started manually thru OpenVPN GUI

ASAP i'll check the log's and the event viewer as you asked.

Tks :)

comment:25 Changed 4 years ago by maverick74

Ok, on location...

1) As before it states (in Services) OpenVPN is set to start automatically on Windows Startup

2) Event Viewer as no logs regarding OpenVPN under "Windows Logs\Application"

a) Did not find any

b) Task Manager states OpenVPN is Running

c) The last change to the log was when i stopped and exit OpenVPN GUI - Before i rebooted the system.

d) If i start OpenVPN GUI it runs with no problems.

e)Funny thing: I can start OpenVPN GUI naturally and it does not report any error (i suppose that if the connection was already established on startup it would alert that a connection was already in use and it does not!)

Also, i don't know if it's suppose to do this, but if a double click openvpnserv the EventViewer? gets an "OpenVPNService error: 0 / StartServiceCtrlDispatcher? failed"... but since i'm not doing the process correctly i suppose this is normal.

comment:26 Changed 4 years ago by xkjyeah

d) What if you try to restart the service? i.e. Services -> OpenVPN Service -> *right click* Restart

It sounds like the .exe file was not installed correctly. If it's alright with you and your company's security policy I can provide you one that writes more debug info to the Event Viewer and/or the Log file.

Drop me an email at daniel.ssq89 at gmail dot com if you'd like to follow up.

comment:27 Changed 4 years ago by MAVERICK74

Hi,

d) Done that a few dozen times already whit no luck. Originally, restarting it solved the problem, but only with version 2.2.2. Actual version didn't work... Anyway, since that was not a real solution, we opted to start the GUI manually. There is, however, a few remaining computers that are in a "win10 pending upgrade" because neither the GUI nor the "Restart" (thru Task Manager > services) are viable options.

Reinstalling the software didn't work either. I think i'll try to edit winregistry and see what i came up whit...

As to a the other solution whit a enhanced .exe (for more debug info), i'll see what i can do, but i won't bet much on that...

comment:28 Changed 4 years ago by xkjyeah

Win10 Pending Upgrade? Are they even Windows 10?

If they aren't Windows 10, try installing .NET framework version 4.

https://www.microsoft.com/en-sg/download/details.aspx?id=17851

comment:29 Changed 4 years ago by MAVERICK74

Hi

OpenVPN Service is having problems (afaik) only on windows 10. In windows 7 it starts ok.

All the machines we've been talking about (and where the errors are) are Windows 10 machines.

I was just saying that we have windows 7 machines that have not been upgraded to windows 10 because it is not viable (on those machines) to manually start OpenVPN GUI, which is what we've been doing on the machines that are running windows 10 (we, manually, start OpenVPN GUI every single time the computer boots)

comment:30 Changed 4 years ago by xkjyeah

@maverick74

It's very odd that the service doesn't start. I have released the x64- and x86-specific binaries, including Debug binaries. In addition, this version checks for both OpenVPN 32- and 64-bit settings when run in a 64-bit environment.

You might want to try the Debug binaries. In addition, if they don't work, then I have to build a console app equivalent to figure out what's going on. As usual, check the Event Viewer for debug messages. You have to refresh (F5) the event viewer after starting/stopping the service to see the latest messages.

In addition, since this isn't exactly an openvpnserv.exe bug, you should file an issue at https://github.com/xkjyeah/openvpnserv2/issues

comment:31 Changed 4 years ago by maverick74

Hi,

I've tried manually cleaning the registry and then do a clean install but got no luck either, so it's not an old key for sure.

I've also tried the debug version as asked, but all i get it the event log is that the service was successfully started... i'm out of clues.

I'll replicate this message and move this discussion to openvpnserv2 (At: https://github.com/xkjyeah/openvpnserv2/issues/3 ).

Last edited 4 years ago by maverick74 (previous) (diff)

comment:32 Changed 3 years ago by Samuli Seppänen

Continuing discussion in openvpnserv2 issue tracker, closing it here.

comment:33 Changed 3 years ago by maverick74

Note: See TracTickets for help on using tickets.