Measurement of IO

[JiaeK]

For the ‘Dream-dashboard’ project (as I mentioned in the last issue), we have also faced the limitation for the measurement of IO.

For instance, we could try by either

• having an API in Dream directly to get the total IO
• or allowing users to hook themselves and listen to the sockets

but it would be great if we could get some recommendations!

[Anton Bachin]

I think measuring I/O would have to be done at a low level. Dream might provide some high-level API to hook into that (or not, if it turns out to be a bad idea), but we'd have to first figure out what low-level code to modify.

Some reasons why this should probably be done at the http/af or lower level:

• Dream receives already parsed request headers from http/af, h2, etc., so some I/O has already been done by the time Dream code is running. It would probably be fragile to try to recompute at the Dream level how many bytes were read while parsing request headers.

• http/af also takes care of inserting chunked encoding delimiters, which add several bytes around each chunk. It's not completely predictable at the Dream level where chunk boundaries were or will be in the actual on-wire streams.

• HTTP/2 (implemented by h2) is a binary protocol, and if we try to recompute the amount of I/O done by h2 while reading requests/sending responses, we would need to know a lot of HTTP/2 details and probably guess a lot.

• There isn't a one-to-one mapping between requests and sockets. HTTP/1 has connection reuse and connection pipelining, while HTTP/2 has multiplexing.

• There are other protocols potentially involved, such as TLS handshakes and TLS itself. Do you want to count raw bytes transmitted, or bytes after handshake and/or within the tunnels?

I think either {http/af, h2, etc.} would have to be modified to "know" how much I/O they are doing, or angstrom/faraday themselves might do that.