Certificate via MI with OpenSSL 1.1.1 errors with "Unknown Padding Type"

[dpb587]

Hello, I believe there's a bug when using the management interface for private keys with openssl 1.1.1 (observed on both Debian and macOS).

When using openssl/1.1.0t , I have been able to provide certificates and keys via management interface. The NEEDS-CERTIFICATE and RSA_SIGN steps are executed successfully.

When using openssl/1.1.1d with the same credentials, it executes the NEEDS-CERTIFICATE , but it errors with the following before ever calling RSA_SIGN .

Fri Sep 20 11:24:28 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]192.0.2.123:1194
Fri Sep 20 11:24:28 2019 Socket Buffers: R=[131072->131072] S=[131072->131072]
Fri Sep 20 11:24:28 2019 Attempting to establish TCP connection with [AF_INET]192.0.2.123:1194 [nonblock]
Fri Sep 20 11:24:29 2019 TCP connection established with [AF_INET]192.0.2.123:1194
Fri Sep 20 11:24:29 2019 TCP_CLIENT link local: (not bound)
Fri Sep 20 11:24:29 2019 TCP_CLIENT link remote: [AF_INET]192.0.2.123:1194
Fri Sep 20 11:24:29 2019 TLS: Initial packet from [AF_INET]192.0.2.123:1194, sid=25d11069 5432dddb
Fri Sep 20 11:24:29 2019 VERIFY OK: depth=1, C=USA, O=Cloud Foundry, CN=openvpn
Fri Sep 20 11:24:29 2019 VERIFY KU OK
Fri Sep 20 11:24:29 2019 Validating certificate extended key usage
Fri Sep 20 11:24:29 2019 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Fri Sep 20 11:24:29 2019 VERIFY EKU OK
Fri Sep 20 11:24:29 2019 VERIFY OK: depth=0, C=USA, O=Cloud Foundry, CN=server
                         vvvv
Fri Sep 20 11:24:29 2019 OpenSSL: error:04066076:rsa routines:rsa_ossl_private_encrypt:unknown padding type
Fri Sep 20 11:24:29 2019 OpenSSL: error:141F0006:SSL routines:tls_construct_cert_verify:EVP lib
                         ^^^^
Fri Sep 20 11:24:29 2019 TLS_ERROR: BIO read tls_read_plaintext error
Fri Sep 20 11:24:29 2019 TLS Error: TLS object -> incoming plaintext read error
Fri Sep 20 11:24:29 2019 TLS Error: TLS handshake failed
Fri Sep 20 11:24:29 2019 Fatal TLS error (check_tls_errors_co), restarting
Fri Sep 20 11:24:29 2019 SIGUSR1[soft,tls-error] received, process restarting
Fri Sep 20 11:24:29 2019 Restart pause, 5 second(s)

Unfortunately, I am a bit out of my depth, but the following seems most notable to me:

• Why does openvpn's behavior change based on whether the certificate came from configuration vs management interface? That is, where an embedded certificate/key in a profile works, if those same values come from management, it fails.
• Recompiling with some additional logging, I'm able to see that it is caused by an explicit RSAerr when the padding does not match the expected RSA_PKCS1_PADDING ( ​ source ).
  • With 1.1.0 it implicitly uses RSA_PKCS1_PADDING via ​ this branch .
  • With 1.1.1 it uses RSA_NO_PADDING via ​ this branch .
  • It seems like RSA_PKCS1_PSS_PADDING instead of RSA_PKCS1_PADDING may be related to TLS 1.3 support?
  • There's a similar-sounding ​ openssl issue about 1.1.1 now always using RSA_NO_PADDING (but the commentary is difficult for me to understand).
  • Possible similar scenario of ​ external private key with TLS 1.3 ?

Any guidance or insight would be appreciated. Thank you for your time.

Reproduction Steps

For the test environment, I built the dependencies with ​ build.sh .

For a smaller repro case, I wrote ​ some Go code to run a management server from an existing, regular OpenVPN profile which contains an embedded cert / key .

Then, assuming some existing profile, the following commands demonstrate the problem:

profile=some/existing/openvpn.ovpn

# 1.1.0 success
~/tmp/padding-bug/management-tool "${profile}" management &
~/tmp/padding-bug/with-openssl-1.1.0/sbin/openvpn <( ~/tmp/padding-bug/management-tool "${profile}" profile )

# 1.1.1 failure
~/tmp/padding-bug/management-tool "${profile}" management &
~/tmp/padding-bug/with-openssl-1.1.1/sbin/openvpn <( ~/tmp/padding-bug/management-tool "${profile}" profile )
...snip...
Fri Sep 20 11:24:29 2019 OpenSSL: error:04066076:rsa routines:rsa_ossl_private_encrypt:unknown padding type
Fri Sep 20 11:24:29 2019 OpenSSL: error:141F0006:SSL routines:tls_construct_cert_verify:EVP lib
Fri Sep 20 11:24:29 2019 TLS_ERROR: BIO read tls_read_plaintext error
Fri Sep 20 11:24:29 2019 TLS Error: TLS object -> incoming plaintext read error
Fri Sep 20 11:24:29 2019 TLS Error: TLS handshake failed
Fri Sep 20 11:24:29 2019 Fatal TLS error (check_tls_errors_co), restarting

[Selva Nair]

This is from OpenSSL 1.1.1 defaulting to PSS padding (for TLS 1.2 and 1.3) which it does internally and requests a non-padded signature. But we currently have an implicit assumption that RSA signature from management will use PKCS1_RSA_PADDING. This makes it impossible to prompt for signature when 1.1.1 is in use.

There is a patch to fix this ( ​ https://patchwork.openvpn.net/patch/587/ ) which is under review and requires some more tweaks.

[dpb587]

Thank you for the prompt reply and explanation. It looks like the patch has been around for a while, so I appreciate you reviving the discussion. If there is anything I can do to help the efforts (probably, namely, feedback or testing) please let me know. In the meantime, I'll track this ticket and the patch [whose number is ironically familiar to my handle :)].

[Gert Döring]

587 has been superseded, but a followup patch was eventually merged into master and 2.5-to-be:

commit 8be444df89be09689ba82e58cbc89452a5fcceca
Author: Arne Schwabe <arne@…>
Date: Wed Dec 4 12:08:36 2019 +0100

Add support for OpenSSL TLS 1.3 when using management-external-key

so you could test that... debian packages are also on the build webserver.

[tct]

We have a similar error here:
​ https://forums.openvpn.net/viewtopic.php?f=6&t=30880#p94141

[becm]

Switching to OpenSSL 1.1.1 and the new padding for TLS 1.2+ affects 3 distinct operation modes:

When OpenSSL handles key operations internally, everything (kind of) just works™ .

The issue linked by tincantech is relate to PKCS#11 hardware tokens, the solution in those cases is to use an up-to-date pkcs11-helper 1.26 (recent OpenVPN 2.5 Windows builds or debian bullseye ).
On Windows there is also ​ CNG for this use case, new padding is ​ supported with OpenVPN 2.5 .

The management protocol is extended in OpenVPN 2.5 to support the new padding type ( this issue ).
The controlling program must also set the version high enough and identify/support the requested padding.