Cannot parse response from Go's `net/http/cgi` package

[Radu Matei]

I am trying to get the following Go program to run in Wagi:

package main

import (
	"fmt"
	"net/http"
	"net/http/cgi"
)

func main() {
	if err := cgi.Serve(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		header := w.Header()
		header.Set("Content-Type", "text/plain; charset=utf-8")

		fmt.Fprintln(w, "Hello, world!")

	})); err != nil {
		fmt.Println(err)
	}
}

Building it with TinyGo:

$ tinygo build -wasm-abi=generic -target=wasi -o main.wasm main.go

Then running it in Wagi (which prints the raw response):

FULL RESPONSE: 
Status: 200 OK
Content-Type: text/plain; charset=utf-8

Hello, world!

ERROR wagi::http_util: HTTP 500 error error=Exactly one of 'location' or 'content-type' must be specified

Not exactly why this is not parsing the response properly.

cc @bketelsen @technosophos

[Radu Matei]

Ah, more context:

DEBUG wagi::handlers: composing response
FULL RESPONSE: 
Status: 200 OK
Content-Type: text/plain; charset=utf-8

Hello, world!

WARN wagi::http_util: corrupt header header=""
DEBUG wagi::handlers: Body(Full(b"Status: 200 OK\r\nContent-Type: text/plain; charset=utf-8\r\n\r\nHello, world!\n"))
ERROR wagi::http_util: HTTP 500 error error=Exactly one of 'location' or 'content-type' must be specified

So it is interpreting the headers as part of the body.

[Radu Matei]

Manually adding a newline between headers and the body works:

package main

import (
	"fmt"
	"net/http"
	"net/http/cgi"
)

func main() {
	if err := cgi.Serve(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		header := w.Header()
		header.Set("Content-Type", "text/plain; charset=utf-8")
		fmt.Fprintf(w, "\n")

		fmt.Fprintln(w, "Hello, world!")

	})); err != nil {
		fmt.Println(err)
	}
}

[Radu Matei]

In the scenario where this is failing (without manually adding the newline), this is how Wagi interprets the response:

b"Status: 200 OK\r\nContent-Type: text/plain; charset=utf-8\r\n\r\nHello, world!\n"

According to the spec:

The response comprises a message-header and a message-body, separated by a blank line.
(https://datatracker.ietf.org/doc/html/rfc3875#section-6.2)

Should this be valid in Wagi?

Relevant implementation in Wagi —

wagi/src/handlers.rs

Lines 144 to 164 in 113749e

 // Okay, once we get here, all the information we need to send back in the response 
 // should be written to the STDOUT buffer. We fetch that, format it, and send 
 // it back. In the process, we might need to alter the status code of the result. 
 // 
 // This is a little janky, but basically we are looping through the output once, 
 // looking for the double-newline that distinguishes the headers from the body. 
 // The headers can then be parsed separately, while the body can be sent back 
 // to the client. 
 debug!("composing response"); 
 let out = stdout_mutex.read().unwrap(); 
 let mut last = 0; 
 let mut scan_headers = true; 
 let mut buffer: Vec<u8> = Vec::new(); 
 let mut out_headers: Vec<u8> = Vec::new(); 
 out.iter().for_each(|i| { 
     if scan_headers && *i == 10 && last == 10 { 
         out_headers.append(&mut buffer); 
         buffer = Vec::new(); 
         scan_headers = false; 
         return; // Consume the linefeed 
     }