Proxy Pass HTTP/1.1 to HTTP/2 forwards invalid header field

[CrazyFluffyPony]

Used software:

https://nginxproxymanager.com/ (they facilitate openresty)
Nginx Version: openresty/1.19.9.1
OS: running inside docker on Linux web-gateway 5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13) x86_64 GNU/Linux

Test case:

• Application with an HTTP service
• Nginx proxying (adds ssl), connects to application via HTTP/1.1 but serves to client via HTTP/2

Problem:

proxy is not removing HTTP/1.1 header fields that are ILLEGAL in HTTP/2
curl output:

$ curl -v --http2 https://xyz.example.org
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 123.123.123.123:443...
* Connected to xyz.example.org (217.92.71.192) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
*  CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2475 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [110 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=xyz.example.org
*  start date: Apr  1 19:50:50 2023 GMT
*  expire date: Jun 30 19:50:49 2023 GMT
*  subjectAltName: host "xyz.example.org" matched cert's "xyz.example.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x1bf79f02cd0)
} [5 bytes data]
> GET / HTTP/2
> Host: xyz.example.org
> user-agent: curl/7.75.0
> accept: */*
>
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [265 bytes data]
* old SSL session ID is stale, removing
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
} [5 bytes data]
* http2 error: Invalid HTTP header field was received: frame type: 1, stream: 1, name: [upgrade], value: [h2]
} [5 bytes data]
* HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
* stopped the pause stream!
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host xyz.example.org left intact
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)

Safari Browsers and Chrome browsers on iOS/MacOS disallow to visit websites with such invalid header fields, Firefox just ignores them and continues working.

NGINX Configuration

server {
  set $forward_scheme https;
  set $server         "123.123.123.123";
  set $port           443;

  listen 80;
listen [::]:80;


listen 443 ssl http2;
listen [::]:443 ssl http2;


  server_name xyz.example.org;


  # Let's Encrypt SSL
  include conf.d/include/letsencrypt-acme-challenge.conf;
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-7/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-7/privkey.pem;






  # Block Exploits
  include conf.d/include/block-exploits.conf;




  # HSTS (ngx_http_headers_module is required) (63072000 seconds = 2 years)
  add_header Strict-Transport-Security "max-age=63072000; preload" always;





    # Force SSL
    include conf.d/include/force-ssl.conf;




proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_http_version 1.1;



  access_log /data/logs/proxy-host-3_access.log proxy;
  error_log /data/logs/proxy-host-3_error.log warn;

client_body_buffer_size 512k;
proxy_read_timeout 86400s;
client_max_body_size 0;
proxy_set_header Host $host;
proxy_hide_header Upgrade;





  location / {





  # HSTS (ngx_http_headers_module is required) (63072000 seconds = 2 years)
  add_header Strict-Transport-Security "max-age=63072000; preload" always;






    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_http_version 1.1;


    # Proxy!
    include conf.d/include/proxy.conf;
  }


  # Custom
  include /data/nginx/custom/server_proxy[.]conf;
}