Opened 5 months ago

Closed 4 months ago

#1392 closed Bug / Defect (duplicate)

OpenSSL: error:C506D064:microsoft cryptoapi:NCryptSignHash:Ungltige Optionen angegeben

Reported by: svnmllr Owned by:
Priority: major Milestone: release 2.5.2
Component: OSS OpenVPN Clients Version: OpenVPN 2.5.0 (Community Ed)
Severity: Not set (select this one, unless your'e a OpenVPN developer) Keywords:
Cc:

Description

Hi,

we got an error message when connecting to our openvpn server:

.\openvpn.exe --config .\config.ovpn --cryptoapicert "THUMB:291C2F01975D895F2C841802E99452E7629C63E5" --verb 11
...
2021-03-10 22:38:39 us=757760 SSL state (connect): TLSv1.3 read server certificate verify
2021-03-10 22:38:39 us=761717 SSL state (connect): SSLv3/TLS read finished
2021-03-10 22:38:39 us=761717 SSL state (connect): SSLv3/TLS write change cipher spec
2021-03-10 22:38:39 us=773724 SSL state (connect): SSLv3/TLS write client certificate
2021-03-10 22:38:39 us=773724 cryptoapicert: enter pkey_rsa_sign_init
2021-03-10 22:38:39 us=777673 cryptoapicert: PSS padding using saltlen = 32
2021-03-10 22:38:39 us=778672 cryptoapicert: calling priv_enc_CNG with alg = SHA256
2021-03-10 22:38:39 us=788676 Signing hash using CNG: data size = 32 padding = 8
2021-03-10 22:38:39 us=789641 SSL alert (write): fatal: internal error
2021-03-10 22:38:39 us=789641 OpenSSL: error:C506D064:microsoft cryptoapi:NCryptSignHash:Ungltige Optionen angegeben
2021-03-10 22:38:39 us=790639 OpenSSL: error:141F0006:SSL routines:tls_construct_cert_verify:EVP lib
2021-03-10 22:38:39 us=793631 TLS_ERROR: BIO read tls_read_plaintext error
...

It seems like the native crypto api throws an "invalid options"-error.
Sadly it is difficult to reproduce because the certificate (& private key) is generated using scep server.
The private key is stored in TPM and not exportable.

We would like to share more details but we do not know which details are relevant.

Best regards,

Change History (5)

comment:1 Changed 5 months ago by Selva Nair

Probably your TPM cannot generate RSA-PSS signature. TPM 2.0 potentially supports it (TPM 1.2 does not) and PSS is required for TLS 1.3. Even with TLS1.2, OpenSSL 1.1.1+ defaults to PSS.

If lack of RSA-PSS in your TPM is the problem, your only option may be to downgrade to TLS 1.1 (--tls_version_max 1.1 in config) or generate a new cert/key in newer hardware / windows cert store.

Version 0, edited 5 months ago by Selva Nair (next)

comment:2 Changed 4 months ago by Gert Döring

Milestone: release 2.5.1release 2.5.2

Is there anything we can do on the OpenVPN side? It does not look like it, so we should then close the ticket.

(Going through all tickets with "milestone 2.5.1", which has already be missed...)

comment:3 Changed 4 months ago by Gert Döring

Owner: plaisthos deleted
Status: newassigned

comment:4 Changed 4 months ago by Selva Nair

This is essentially the same as #1296. See the discussion there.

comment:5 Changed 4 months ago by Gert Döring

Resolution: duplicate
Status: assignedclosed

So I'll close this as DUP of #1296.

(I thought it looked familiar but didn't go out searching. Thanks for the pointer)

Note: See TracTickets for help on using tickets.