TLS1.3: p11prov_DeriveKey:Host out of memory

[kshitizvars]

Hi @simo5

We are able to run ECDH key exchange operations on tls1.2, but facing some issues in tls1.3.

ERROR
2070FEF7FFFF0000:error:40800002:pkcs11:p11prov_DeriveKey:Host out of memory error:/usr/src/debug/pkcs11-provider/0.3/src/interface.gen.c:1011:Error returned by C_DeriveKey
2070FEF7FFFF0000:error:0A0C0103:SSL routines:ssl_derive:internal error:ssl/s3_lib.c:4901:
shutting down SSL

Program received signal SIGSEGV, Segmentation fault 

On further debugging, it is because of wrong client public key point (EC_POINT) and its length:-

TLS1.3 logs

(gdb) p *((P11PROV_OBJ *)0x587920).attrs
$11 = {type = 16462464871219690244, pValue = 0xeed50f6aa03e120d, ulValueLen = 14520826335356353610}
(gdb) p *((P11PROV_OBJ *)0x587920)->attrs
$12 = {type = 16462464871219690244, pValue = 0xeed50f6aa03e120d, ulValueLen = 14520826335356353610}

TLS1.2 logs

(gdb) p *(CK_ATTRIBUTE *) 0x588b20
$24 = {type = 2152681555, pValue = 0x5ca280, ulValueLen = 65}
(gdb) p ((P11PROV_OBJ *)0x587920)->attrs[0]
$25 = {type = 2152681555, pValue = 0x5ca280, ulValueLen = 65}
(gdb) p ((P11PROV_OBJ *)0x587920)->attrs[1]

Do you have any comments?
Debug logs:-
debug_tls1_2.log
debug_tls1_3.log

A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Openssl configuration:-

[openssl_init]
providers = provider_sect
alg_section = algorithm_sect
 
# List of providers to load
[provider_sect]
default = default_sect
pkcs11 = pkcs11_sect
 
[default_sect]
activate = 1
[pkcs11_sect]
module = /usr/lib/ossl-modules/pkcs11.so
pkcs11-module-path = /usr/lib/libckteec.so.0
pkcs11-module-cache-keys = false
pkcs11-module-quirks = no-operation-state
pkcs11-module-block-operations = digest
activate = 1
 
[algorithm_sect]
default_properties = ?provider=pkcs11

2. server side command:-

$ openssl s_server -key "pkcs11:model=OP-TEE%20TA;manufacturer=Linaro;serial=0000000000000001;token=token0;id=%01;object=ecc-key-256;type=private?pin-value=1234" -cert server.crt -accept 443

3. Run openssl s_client from another machine:-
   #Client side command:-
   $ openssl s_client -connect <server ip>:443 -tls1_3 -ciphersuites 'TLS_AES_256_GCM_SHA384' -curves secp256r1

Expected behavior
TLS1.3 connection should work fine with key exchange operation.

Operating environment (please complete the following information):

• OS: Linux

Does this mean, we haven't tested ECDH derive with provider in TLS1.3?

[kshitizvars]

Hi @simo5

Are you planning to merge this commit simo5@75cc2c3 to main branch?

[simo5]

@kshitizvars I guess I should eventually, I had forgotten I written that code ...
If I have not opened a PR it must mean that code was not ready yet.

[kshitizvars]

Hi @simo5

FYI, I have tested simo5@75cc2c3 patch after running TLS1.2 & TLS1.3.
Initially, I was getting the issue mentioned in the discussion (DeriveKey: Host out of memory) but after applying simo5#2, no more issues and TLS1.2 & TLS1.3 connections are working fine with ECDH curve as secp256r1.