PemWriter creates an invalid PEM file when byte begin with a zero

Question

PemWriter creates an invalid PEM file when byte begin with a zero

anupkattel opened this issue 4 years ago · comments

Initially I found an intermittent issue in my integration tests with RSAParameters to PEM conversion using PemWriter and reading it back using PemReader.

Now I have narrowed it down to PemWriter and I can replicate it.

When any of the RSAParameters such as Modulus, D, P, etc start with zero as the first byte, PemWriter generates an invalid PEM file. It does not include the first zero in the PEM file.

I am looking further into this. I will post if I find anything interesting.

best-part-no-part · Answer 1 · Sun Apr 12 2020 10:26:41 GMT+0800 (China Standard Time)

All initial zeros in the byte arrays of RSAParameters are lost when we convert them to big integers.

        public void WritePrivateKey(RSAParameters parameters)
        {
            var sequence = new DerAsnSequence(new DerAsnType[]
            {
                new DerAsnInteger(ToBigInteger(new byte[] { 0x00 })),   // Version
                new DerAsnInteger(ToBigInteger(parameters.Modulus)),
                new DerAsnInteger(ToBigInteger(parameters.Exponent)),
                new DerAsnInteger(ToBigInteger(parameters.D)),
                new DerAsnInteger(ToBigInteger(parameters.P)),
                new DerAsnInteger(ToBigInteger(parameters.Q)),
                new DerAsnInteger(ToBigInteger(parameters.DP)),
                new DerAsnInteger(ToBigInteger(parameters.DQ)),
                new DerAsnInteger(ToBigInteger(parameters.InverseQ))
            });

Unless we know the length of the array by some means such as knowing RSA Key size, I'm not sure how DerAsn can tell the real length.

best-part-no-part · Answer 2 · Sun Apr 12 2020 11:46:27 GMT+0800 (China Standard Time)

Turns out that the problem is not in the export, it's the import that doesn't work well.

I used .net core's inbuilt PKS#1 exporter and tried exporting a certificate that doesn't get exported back by PemUtils properly. The exported PEM file are exactly the same from both methods. So, it must be the import that is not taking care of the leading zero bytes.

huysentruitw · Answer 3 · Sun Apr 12 2020 12:27:41 GMT+0800 (China Standard Time)

Thanks for reaching out. Can you post the RSA parameters you're testing with? Just to make sure we'll do the same testing...

best-part-no-part · Answer 4 · Sun Apr 12 2020 14:38:23 GMT+0800 (China Standard Time)

Sure. I basically ran a loop 100 times and kept generating new RSAParameters and trying to export to PEM and import back until it failed. I noticed that every time it fails, one of the RSA parts starts with zero byte. For example, D or Private Exponent is like byte[] {0, .......}

Here's a valid PEM file that works on this website: https://8gwifi.org/PemParserFunctions.jsp

If you try importing this certificate, you will see that the Private Exponent or D will be of 63 bytes instead of being 64 as usual (64 bytes because I generated a 512 bit key for simplicity)

best-part-no-part · Answer 5 · Sun Apr 12 2020 15:00:06 GMT+0800 (China Standard Time)

Here's another one where InverseQ or the coefficient starts with a zero:

huysentruitw · Answer 6 · Tue Apr 14 2020 03:19:13 GMT+0800 (China Standard Time)

In short, the 0x00 is used as padding in case the second byte is > 0x80 (sign-bit) so the number isn't being treated as a negative integer. However, sometimes the 0x00 should be stripped before putting the byte array into RSAParameters and in other cases the 0x00 must be kept, and I don't see when or why.

For reference, here is the openssl code that manages the ASN INTEGER type: https://github.com/openssl/openssl/blob/e7fb44e7c3f7a37ff83a6b69ba51a738e549bf5c/crypto/asn1/a_int.c

Marc-André Moreau · Answer 7 · Thu Oct 28 2021 03:48:19 GMT+0800 (China Standard Time)

We're hitting the same problem, we've been wondering why some keys were not loading properly, and it turns out it's because of leading zeros being omitted. The quickest workaround is to generate and load keys until one of them loads properly, but we should really fix the encoding to encode leading zeros.