ruby / openssl

This will sometimes fail:

30_000.times do
  key = OpenSSL::PKey::EC.generate('prime256v1')
  expect(key.private_key.to_s(2).bytesize).to eq(32)
end

Why the private key is only 31 bytes sometimes? Is it a bug?

Related: pushpad/web-push#4

This isn't a bug of openssl gem. EC key's private parameter d is a integer, represented as a OpenSSL::BN and it does not know the expected length in byte, because it can handle arbitary size of integers (bignum).

And OpenSSL::BN#to_s(2) calls BN_bn2bin which do not add padding.

openssl/ext/openssl/ossl_bn.c

Line 354 in 79aa330

if (BN_bn2bin(bn, (unsigned char *)RSTRING_PTR(str)) != len)

For this use case, you need to add padding as needed according to Section 2.3.7 of SEC1, and you also may refer to Appendix C.4 of SEC1.

If you're unsure how to properly handle these numbers, I'd strongly recommend to use DER or PEM format (to_der, to_pem) to export and import key materials.

@sorah Thanks for the reply!

Unfortunately I am not a cryptography expert, but I am trying to fix a code written by others in an open source product.

This test in particular sometimes (rarely) fails:

https://github.com/pushpad/web-push/blob/20f40d9586b524b8c75e64887683931d6e11b078/spec/web_push/vapid_key_spec.rb#L19

Can I fix that test by expecting a length of 31..32 bytes? Or it may return other lengths?

Or that test doesn't make sense at all?

It depends on the usage of the method output. You may need to return fixed length string, or you may not.

@sorah I am not asking about our specific use case. I'll try to rephrase the question:

key = OpenSSL::PKey::EC.generate('prime256v1')
length = key.private_key.to_s(2).bytesize
length == 31 || length == 32

Is the last statement always true?

No. The private key is an integer anywhere between 1...key.group.order so the OpenSSL::BN#to_s(2)'s encoding could be as short as 1 octet long.

Yes, it is not guaranteed to be 31 or 32 bytes string.

I'd repeat:

It depends on the usage of the method output.

It depends how your private_key method result is used for. You may need 32-bytes fixed length string or you may not.

I'd strongly recommend to simply expose OpenSSL::PKey::EC object or PEM/DER formatted string to keep things simple and rely on well-trusted(?) implementation. This conversation shows you still don't understand how EC works.

Otherwise do roundtrip test (try export and load into OpenSSL::PKey).

So I understand that private key is an integer and when your try to convert it into a string of 0 and 1 (using to_s(2)) you can get different string lengths based on the actual integer.

Is the same true for the public key? Because I see another test (not written by me) that again tests on the length and I wonder if that is also wrong.

OpenSSL::PKey::EC private key length should be 32 bytes, but sometimes is only 31 bytes