segmentation fault might occur when SSL renegotiation happens

[Alex Zhang]

Hello!

Recently I found one of our Nginx worker process exited abnormally (segmentation fault). The backtrace is like:

nginx: worker process(ngx_http_lua_ssl_cert_handler+0x1c2) [0x5b8313]
nginx: worker process(tls_post_process_client_hello+0x1be) [0x7302e9]
nginx: worker process(ossl_statem_server_post_process_message+0x6c) [0x72dd0f]
nginx: worker process(read_state_machine) [0x71b25d]
nginx: worker process(state_machine) [0x71abc1]
nginx: worker process(ossl_statem_accept+0x1d) [0x71a74d]
nginx: worker process(ssl3_read_bytes+0xed4) [0x6e9846]
nginx: worker process(ssl3_read_internal) [0x6f33e9]
nginx: worker process(ssl3_read+0x38)
nginx: worker process(ssl_read_internal+0x165)
nginx: worker process(SSL_read+0x5b)
nginx: worker process(ngx_ssl_recv+0xc5)
nginx: worker process(ngx_http_v2_read_handler)
nginx: worker process(ngx_http_v2_idle_handler+0x116)
nginx: worker process(ngx_epoll_process_events)
nginx: worker process(ngx_process_events_and_timers+0xd3)

The coredump point is inside function ngx_http_lua_ssl_cert_handler.

You can reproduce this problem by the following way.

server {
    listen 8443 ssl http2;
    ssl_certificate_by_lua_block {
        return;
    }
     ssl_certificate /path/to/cert;
     ssl_certificate_key /path/to/pkey;
     location / {
          return 200;
     }
}

openssl s_client -connect 127.0.0.1:8443 -alpn h2 -reconnect

Then type "R" in the interactive mode (trigger the TLS reneogatitaion).

ALPN protocol: h2
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 8C3B8082AEEDC0AF9894C28A7ED50F2B52AA5C2274B4E15509DFEA097B12CB42
    Session-ID-ctx:
    Master-Key: DABD0D31B6A799DB05E199DC89045833480E47EF8F43DC2DDD36A4851C2026DB05A242580928135214395734343EADF7
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 92 bf 74 6e 7f 3d 7a 0b-34 4a 48 18 11 7f 60 31   ..tn.=z.4JH...`1
    0010 - 1e 9b 46 8f 11 62 d6 b2-83 ea 2e d5 45 26 a6 46   ..F..b......E&.F
    0020 - a5 c8 13 48 35 0b 87 fd-26 cd 6e aa c1 0a 35 c9   ...H5...&.n...5.
    0030 - 1b 2f 93 5a d6 9d 11 b6-c2 5a df b0 53 23 b3 b4   ./.Z.....Z..S#..
    0040 - de d8 73 50 b6 61 e5 1b-08 b9 9b b4 22 18 cc e0   ..sP.a......"...
    0050 - 3a 13 c0 d0 db f1 7e 3f-2a 01 14 61 4c 08 0e 4c   :.....~?*..aL..L
    0060 - 63 cb 02 93 92 80 03 64-33 c9 aa b0 d8 63 c8 33   c......d3....c.3
    0070 - 92 b5 c9 4c 35 65 fd 62-f0 fd 53 cf 0b 2f 97 c6   ...L5e.b..S../..
    0080 - b7 bc f3 e0 83 15 3a 94-1a 95 78 68 33 99 dc df   ......:...xh3...
    0090 - 67 b5 e6 4a 16 ca 20 e1-43 85 eb fc 6a 52 01 d7   g..J.. .C...jR..
    00a0 - 9b 51 18 b3 e1 3d 32 fd-80 a3 4e 40 7c 45 7f 0b   .Q...=2...N@|E..
    00b0 - 52 06 b8 07 7e ee c6 d3-16 8b df b4 24 f7 f8 2c   R...~.......$..,
    00c0 - 58 0f e9 86 6a a3 84 70-0e 08 87 b0 84 22 51 a5   X...j..p....."Q.

    Start Time: 1532434389
    Timeout   : 7200 (sec)
    Verify return code: 10 (certificate has expired)
    Extended master secret: yes
---
R
RENEGOTIATING
write:errno=0`

Now open Nginx's error log and we can see:

2018/07/24 19:13:25 [alert] 7573#7573: worker process 7574 exited on signal 11 (core dumped)

After analysis, I found the reason finally. This problem occurs when client performed the SSL renegotiation and the current connection is already upgraded to HTTP/2, furthermore, current connection should reuse a TLS session.

When TLS renegotiation happens, the ngx_http_lua_ssl_cert_handler will be called.

Since current connection reused a TLS session, so the cctx inside this function is still NULL, the following if block will not be executed:

 if (cctx && cctx->entered_cert_handler) {
        /* not the first time */

        if (cctx->done) {
            ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0,
                           "lua_certificate_by_lua: cert cb exit code: %d",
                           cctx->exit_code);

            dd("lua ssl cert done, finally");
            return cctx->exit_code;
        }

        return -1;
    }

Then c->data will be treated as the ngx_http_connection_t, we know when the functionngx_http_v2_init called, the c->data will be set to ngx_http_v2_connection_t (rather than the original ngx_http_connection_t). While ngx_http_lua_ssl_cert_handler doesn't distinguish this situation. It just uses the "ngx_http_v2_connection_t" as "ngx_http_connection_t", and some invalid address will be referenced.

[Alex Zhang]

I have submitted a PR for the fixup: #1355.

[Alex Zhang]

ping @agentzh .

[kwanhur]

couldn't reproduce it with nginx-1.13.6+lua-nginx-module-0.10.12rc2+openssl-1.1.0g

[Alex Zhang]

@kwanhur What about openssl-1.1.0h? I can reproduce this problem through OpenResty/1.13.6 built with OpenSSL/1.1.0h.