Add Hybrid Public Key Encryption (HPKE) API Support

[sylph01]

This patch introduces Hybrid Public Key Encryption (HPKE; RFC 9180) through OpenSSL's HPKE APIs ( https://docs.openssl.org/3.5/man3/OSSL_HPKE_CTX_new/ ), added in OpenSSL 3.2.0.

Usage

suite = OpenSSL::HPKE::Suite.new_with_names(:dhkem_x25519_hkdf_sha256, :hkdf_sha256, :aes_128_gcm)
pkey = OpenSSL::HPKE.keygen_with_suite(suite)
pub = pkey.raw_public_key
s = OpenSSL::HPKE::Context::Sender.new(:base, suite)
enc = s.encap(pub, "info")
ct = s.seal("aad", "hi")
r = OpenSSL::HPKE::Context::Receiver.new(:base, suite)
r.decap(enc, pkey, "info")
puts "roundtrip: #{r.open("aad", ct) == "hi"}, export match: #{s.export(32,"l")==r.export(32,"l")}"

APIs

OpenSSL::HPKE::Suite

• new: Instantiate cipher suite with KEM, KDF, and AEAD identifiers listed in RFC 9180
• new_with_names: Instantiate cipher suite with pre-defined names. Uses the list of KEMs, KDFs, AEADs listed in RFC 9180.

OpenSSL::HPKE

• keygen: Generate OpenSSL::PKey private key with the specified KEM, KDF, and AEAD ID.
  • This exposes OpenSSL's OSSL_HPKE_keygen() API.
• keygen_with_suite: Generate OpenSSL::PKey private key with the specified cipher suite

These are more like utility functions so if they look extraneous they can be removed in favor of using OpenSSL::PKey to generate corresponding keys.

OpenSSL::HPKE::Context::Sender and OpenSSL::HPKE::Context::Receiver

• new: Instantiate HPKE Context.
  • Currently supports :base mode only; I wanted to let the maintainers see this pull request before adding :auth, :psk, and :auth_psk modes

OpenSSL::HPKE::Context::Sender

• encap: Encapsulates key into the specified public key. Takes receiver's public key and info (application context information)
• seal: Using the encapsulated key, seal message into ciphertext. Takes aad (additional authenticated data) and ciphertext itself.

OpenSSL::HPKE::Context::Receiver

• decap: Decapsulates the key using the private key. Takes the encapsulation, private key, and info (application context information).
• open: Using the decapsulated key, decrypt the ciphertext. Takes aad and ciphertext.

Availability

• This functionality is available on OpenSSL newer than 3.2.0 without FIPS mode.
  • As OpenSSL's FIPS mode does not implement EC KEMs, HPKE on OpenSSL is unavailable even for curves that are supported by FIPS.
• LibreSSL, AWS-LC is not supported.
  • As far as I know, they do not have the corresponding APIs.

[rhenium]

Thanks for working on this!

Regarding the Ruby API:

OpenSSL::HPKE::Suite

* `new`: Instantiate cipher suite with KEM, KDF, and AEAD identifiers listed in RFC 9180

* `new_with_names`: Instantiate cipher suite with pre-defined names. Uses the list of KEMs, KDFs, AEADs listed in RFC 9180.

Do you think if it makes sense to have this as an overload to .new instead of a separate method?

I'm ambivalent about maintaining our own name list in ruby/openssl. I wonder if we could use OSSL_HPKE_str2suite() (https://docs.openssl.org/master/man3/OSSL_HPKE_CTX_new/#protocol-convenience-functions).

OpenSSL::HPKE

* `keygen`: Generate `OpenSSL::PKey` private key with the specified KEM, KDF, and AEAD ID.
  
  * This exposes OpenSSL's `OSSL_HPKE_keygen()` API.

* `keygen_with_suite`: Generate `OpenSSL::PKey` private key with the specified cipher suite

These are more like utility functions so if they look extraneous they can be removed in favor of using OpenSSL::PKey to generate corresponding keys.

This seems useful to me, since there is no straightforward way to map from HPKE KEM IDs to OpenSSL algorithm object names.

However I'm less sure about .keygen method taking a triple of IDs. Since it would most likely be used together with OpenSSL::HPKE::Context::{Sender,Receiver}.new, which requires an instance of Suite anyway, perhaps the variant taking a Suite would be sufficient?

OpenSSL::HPKE::Context::Sender and OpenSSL::HPKE::Context::Receiver

* `new`: Instantiate HPKE Context.
  
  * Currently supports `:base` mode only; I wanted to let the maintainers see this pull request before adding `:auth`, `:psk`, and `:auth_psk` modes

That sounds like a good plan to me.

[rhenium]

For style, please add newlines at the end of files and break long lines.

[rhenium]

The list in RFC 9180 appears to be the initial content of the registry and I'd expect OpenSSL to implement new KEMs in the future, once published as an RFC: https://www.iana.org/assignments/hpke/hpke.xhtml

Is there any way to avoid hardcoding the maximum size?

[sylph01]

I looked for any publicly available constants/functions that can reach Npk, but couldn't find one.

• OSSL_HPKE_get_public_encap_size(suite) does give Nenc, but for classical KEMs we can assume Nenc = Npk, but for PQ algorithms we cannot assume that (for example, ML-KEM has Nenc of 768 but has Npk of 800). And as for cases when Nenc =/= Npk, it is Nenc < Npk, which means we cannot use the size of Nenc for the buffer size of key generation.
• There is OSSL_HPKE_MAXSIZE (512 for current OpenSSL implementations) under hpke.c, which is the internal limit for how large this buffer can be, but that value is not exposed publicly (only local to hpke.c).
• EVP_PKEY_get_size(pkey) exists but that needs a concrete EVP_PKEY, which means we cannot use that for deciding the buffer size of key generation.

As this function is the only part with hardcoded buffers, when PQ suites are introduced into HPKE, only this part will break, but breaks by raising that we could not keygen (EVP_PKEY_get_octet_string_param(skR, ENCODED_PUBLIC_KEY, pub, *publen, publen is called, but with the current publen it returns 0 without overwriting), not by introducing a buffer overflow.

For the reasons stated above, I suggest re-writing the comment so that:

• The value 133 is the maximum Npk for classical KEMs
• When a suite that requires a larger Npk is provided, this function will raise an error
• When PQ KEMs are introduced in future versions of OpenSSL this value needs to change accordingly

If we could get Npk in a similar manner with OSSL_HPKE_get_public_encap_size(suite) we can migrate gracefully, but as far as my research goes we're stuck with this option for now.

I might as well write up an issue in openssl/openssl...

[rhenium]

Thanks for the explanation. I saw openssl/openssl#31205, and it does seem like a missing feature in the current OpenSSL API. Let's revisit this when OSSL_HPKE_get_public_key_size() becomes available.

[sylph01]

Regarding the public API:

Do you think if it makes sense to have this as an overload to .new instead of a separate method?
I'm ambivalent about maintaining our own name list in ruby/openssl. I wonder if we could use OSSL_HPKE_str2suite() (https://docs.openssl.org/master/man3/OSSL_HPKE_CTX_new/#protocol-convenience-functions).

Using OSSL_HPKE_str2suite() makes sense. As EVP_PKEY takes an algorithm name (instead of internal identifiers), it would make sense to just use algorithm names instead of using identifiers, and make the identifiers completely internal. In that case, the public API will accept only algorithm names, and Ruby users wouldn't have to write their own conversion from names to identifiers.

This seems useful to me, since there is no straightforward way to map from HPKE KEM IDs to OpenSSL algorithm object names.
However I'm less sure about .keygen method taking a triple of IDs. Since it would most likely be used together with OpenSSL::HPKE::Context::{Sender,Receiver}.new, which requires an instance of Suite anyway, perhaps the variant taking a Suite would be sufficient?

This looks reasonable, so I will change keygen to accepting a Suite.

[sylph01]

• 36c705e -> 9b0d119 changes the instantiation of Suite and interface of keygen (according to the comment above)
• 30905ca should have fixed the styling
• The release that happened yesterday was a security patch release so it didn't ship with additional methods that we want, so I suggest deferring that to a different PR. I can add a TODO comment if needed.

[rhenium]

Do you think if it makes sense to have this as an overload to .new instead of a separate method?
I'm ambivalent about maintaining our own name list in ruby/openssl. I wonder if we could use OSSL_HPKE_str2suite() (https://docs.openssl.org/master/man3/OSSL_HPKE_CTX_new/#protocol-convenience-functions).

Using OSSL_HPKE_str2suite() makes sense. As EVP_PKEY takes an algorithm name (instead of internal identifiers), it would make sense to just use algorithm names instead of using identifiers, and make the identifiers completely internal. In that case, the public API will accept only algorithm names, and Ruby users wouldn't have to write their own conversion from names to identifiers.

Just to clarify, I was wondering whether .new could handle both algorithm IDs (in Integer) and OpenSSL nicknames (in String) by checking types internally, instead of providing a separate method for each.

OpenSSL::HPKE::Suite#*_id already exposes the integer IDs, so accepting it in .new is not unreasonable. That said, I don't currently have a use case for this.

[sylph01]

Just to clarify, I was wondering whether .new could handle both algorithm IDs (in Integer) and OpenSSL nicknames (in String) by checking types internally, instead of providing a separate method for each.

Oops, I went too far on that one. I checked if there's an actual use case that wants the ID instead of the name:

• Messaging Layer Security defines a subset of HPKE suites (RFC 9420 Section 17.1), which means it doesn't expose the KEM/KDF/AEAD IDs directly.
  • For this reason I used names to instantiate the cipher suite in my MLS implementation, and this is what I had in mind writing that part.
• TLS Encrypted ClientHello's ECHConfig specifies the KEM/KDF/AEAD IDs directly (RFC 9849 Section 4), so having the option to specify integer IDs is not just convenient, it's more like necessary.

As such I will fix that on the next round of patches.

[rhenium]

Please add rdoc comments.

Other than docs and the minor comments below, this looks good to me.

[rhenium]

In tests, we should check for openssl?(3, 2, 0) to avoid accidentally skipping tests for OpenSSL versions that are known to support it.

[sylph01]

Fixed @ cfcf9f2.
I am split on this one though; it's fine for now, but if LibreSSL gets HPKE, this version gate becomes more complicated, and I think checking the existence of the HPKE module would become the more straightforward check. If the point is to guard against the version gate (checking of OSSL_HPKE_CTX_new) not functioning properly on versions that actually have HPKE support, then I'll keep this version.

[sylph01]

Added documentation @ 8c05446.

I'll come back to the following when the right time comes:

• Fixing the hard-coded Npk on ossl_hpke_keygen(): after OSSL_HPKE_get_public_key_size() lands in OpenSSL upstream
• Adding PSK/Auth/AuthPSK modes: I will come back to adding PSK mode when an RFC that uses that is published (likely COSE-HPKE). I am yet to find use cases for Auth and AuthPSK modes.

(I have another feature regarding recent RFCs that I want to land so I'll work on that next)

[rhenium]

Could you squash changes into a commit? The changes look good to merge to me.

[sylph01]

ready @ 6b80681

[rhenium]

Thank you for your work (and patience with me) on this!