[tokio-rustls] Offer a `readable()` method like TcpStream
horazont opened this issue · comments
The rustls (0.20.x) documentation states that the number of plaintext bytes which can currently be read can be obtained at any time by calling process_new_packets.
Presumably, this would allow to provide a readable() method (like it exists on TcpStream).
The use case here is an application which will have north of one thousand mostly idle connections. Keeping receive buffers around for each of them to pass to read_buf is very inefficient: Instead, it is desirable to wait for the socket to actually become readable before allocating a buffer to receive into.
This sounds reasonable, but there is no need to add new interfaces to implement it.
You can improve it like this
let buf = poll_fn(|cx| {
let mut buf = alloc_buf_from_pool();
match tls_stream.poll_read(&mut buf) {
Poll::Ready(Ok(_)) => Poll::Ready(Ok(buf)),
Poll::Ready(Err(err)) => {
free_buf_to_pool(buf);
Poll::Ready(Err(err))
},
Poll::Pending => {
free_buf_to_pool(buf);
Poll::Pending
}
}
}).await?Interesting idea. I don't like about this that it'll have a pointless temporary allocation every time (socket is not readable -> alloc+free+pending -> readable -> alloc+poll_read+ready -> called again for next read -> alloc+free+pending: two allocations per read). Waiting for readable would avoid that by only allocating once there is a reasonable chance that the socket is indeed ready for reading.
(Even better would be to also offer try_read, for what it's worth.)
It is much better if you don't treat it as a pointless alloc, but instead treat it as an optimistic alloc. we can be optimistic that the first access to the stream is readable.
But for pessimistic scenarios, it makes sense to add a readable().
We can implement it without add new interface, but I think it would be nice to add a convenience method.
if tls_stream.get_ref().1.wants_read() {
tls_stream.get_ref().0.readable().await?;
}
let mut buf = alloc_buf_from_pool();
tls_stream.read(&mut buf).await?;
I don't think that last one covers decoded cleartext bytes still available. You could implement readable() by first checking if there's decoded bytes, and if there are not await for the underlying stream to become readable, read data (this requires a buffer in the TLS stack too! But not on the application side), and if its enough to get cleartext unblock. Its however tricky, because even if socket data is received it might not have been enough to decode a TLS frame.
Yes, but it can be applied to 90% of the scenes.
A better solution need add a readable() method to tokio_rustls::common::Stream.
It is much better if you don't treat it as a pointless alloc, but instead treat it as an optimistic alloc. we can be optimistic that the first access to the stream is readable.
Well, these connections are both long lived and idle most of the time (XMPP server-to-server connections), and when they are not, oftentimes only an application-level keepalive/ping is sent (small) or a single short stanza. So I'd ideally be able to readable().await and then ask the socket (either via the FIONREAD ioctl if plain TCP or via the process_new_packets method in rustls) how many bytes are available, allocate a matching buffer (up to a limit) and read only that much. Otherwise I'm forced to pick between always using a small buffer (which gets inefficient throughput-wise when a bulk of data comes in, which is not easy, if at all possible, to know in advance) or always using a large buffer (which is very inefficient memory-wise if I have to keep it allocated all the time) or do the frequent reallocations (which is, indeed, pointless, because it's going to be rather pessimistic).
Hence… readable(). I'd be happy to take a shot at that once the rustls 0.20 support (which as I saw you are already working on) is there.
Caching and somehow exposing the last seen rustls::IoState seems sensible to me.
So it turns out that supporting this without hard-depending on readable() in IO (for which there isn't even a trait in tokio) is quite tricky.
I was able to implement it downstream with the pattern suggested in #75 (comment) though.
From my perspective, this issue could be closed for now, unless anyone else wants to keep it open for tracking purposes.