Opened 3 years ago

Closed 2 years ago

#1294 closed Bug / Defect (worksforme)

Centos7 openvpn seems not to free unused memory

Reported by: gmazzini Owned by:
Priority: critical Milestone: release 2.4.9
Component: plug-ins / plug-in API Version:
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords:


An openvpn server 2.4.9 running on CentOS7 seems so eager for *free* memory as to have its plugin suddenly starting to fail authentication on any further connection (or re-connection) as soon as system *free* memory gets close to a few megabytes. Then, openvpn.log says

PLUGIN_CALL: POST /usr/lib64/openvpn/plugins/ status=1

Then only killall -HUP openvpn gets all established connections shutted down and immediately recovered, plus a fresh new bunch of "free memory". Same behaviour when is compiled from sources (in place of Debian distribution's one).
Might all that depend on some trouble in memory management?

Some more details:

  • System: AWS EC2 host with Linux 3.10.0-1127.8.2.el7.x86_64, CentOS Linux release 7.8.2003 (Core).
  • OpenVPN Version (from its CLI Management console):

OpenVPN 2.4.9 x86_64-redhat-linux-gnu [Fedora EPEL patched] [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Apr 24 2020

  • Current status:

[a]. 380 clients (from openvpn-status.log, CLIENT_LIST)
[b]. running processes:

nobody   25417  0.4  2.0 250540 77488 ?        Ss   giu18   7:04 /usr/sbin/openvpn --status /run/openvpn-status.log --status-version 2 --config openvpn.conf
root     25418  0.4  7.8 501452 299324 ?       S    giu18   5:58  \_ /usr/sbin/openvpn --status /run/openvpn-status.log --status-version 2 --config openvpn.conf

[c]. total openvpn memory (from pmap -X 25417 25418):

         Address Perm   Offset Device    Inode   Size    Rss    Pss Referenced Anonymous Swap Locked Mapping
                                               ======  =====  ===== ========== ========= ==== ======
25417:   /usr/sbin/openvpn                     250604  77556  74640      77556     73580    0      0 KB
25418:   /usr/sbin/openvpn                     753712 378532 373521     377936    372072    0      0 KB

with total [heap] ~ 389MB

[d]. free -h

              total        used        free      shared  buff/cache   available
Mem:           3,7G        606M        784M        120M        2,3G        2,7G
Swap:            0B          0B          0B

Thank you very much,
giacomo mazzini

Change History (2)

comment:1 Changed 3 years ago by Gert Döring

25418 is likely the forked-plugin-stays-root process.

My openvpn server with plugin-auth-pam does not exhibit this, and this is the first we hear about it - so it hints at a problem with one of the PAM modules loaded. Which PAM modules are you using?

I use, and memory is not growing here

root    60706    3.9  0.1  26832 22784  -  Ss   14Mar20    4428:17.64 /usr/local/sbin/openvpn --cd /usr/local/etc/openvpn --daemon openvpn --config server.conf
root    60707    0.0  0.0  23312 15256  -  I    14Mar20       5:48.91 /usr/local/sbin/openvpn --cd /usr/local/etc/openvpn --daemon openvpn --config server.conf

(now, this is a FreeBSD system and no Linux, but if plugin-auth-pam itself leaks memory it should be visible here too)

comment:2 Changed 2 years ago by Gert Döring

Resolution: worksforme
Status: newclosed


I am closing this now, as I can't do anything on a "critical" bug where the reporter isn't talking to me.

Generally speaking it seems some PAM module is leaking memory, and that is nothing OpenVPN can influence or fix. BUT: you can use the plugin-auth-pam module from git master / 2.5_beta2, which can do async/deferred authentication, which means "all PAM handling will be done in a new process for each connection".

So it does not matter how much memory the PAM stuff uses, as it's only short-lived.

(You need to add setenv deferred_auth_pam to your server config to activate the new behaviour)

Note: See TracTickets for help on using tickets.