Add a streaming interface
opened this issue · comments
ece_decrypt
and ece_encrypt
operate on the full plaintext and ciphertext. This is fine for Web Push, which shouldn't be used for large messages, but we'll want to think about a streaming API for other uses. Here's a sketch of how such an API might look:
// Create a stream with a buffer size of 8k.
ece_decrypt_stream_t* s = ece_decrypt_stream_new(ECE_SCHEME_AES128GCM, 8192);
ece_buf_t ciphertext;
// Read encrypted blocks from a file, socket, libuv stream, etc.
size_t written = ece_decrypt_stream_write(s, &ciphertext);
if (written < ciphertext.length) {
// Once the buffer is full, or an error occurs, read the decrypted data.
// `chunk` is a slice of the stream's buffer, not a copy. This avoids an
// intermediate copy if we want to write the plaintext to another stream,
// but does mean we'll need to copy the slice if we want to keep the plaintext
// in memory.
ece_buf_t chunk;
int err = ece_decrypt_stream_read(s, &chunk);
if (err) {
// Decryption failed. Close the source stream and free `s`.
ece_decrypt_stream_free(s);
}
} else {
// We can keep writing to `s`.
}
// Once we've written all decrypted data from the source stream, flush all
// data remaining in the buffer. Returns true if we need to read from `s` again.
if (ece_decrypt_stream_flush(s)) {
ece_buf_t lastChunk;
int err = ece_decrypt_stream_read(s, &lastChunk);
if (err) {
ece_decrypt_stream_free(s);
}
}
Another idea: buffer on record boundaries (rs
) instead of specifying a buffer size. The disadvantage of that is it'll result in lots of allocations if the sender picks a small rs
. With my first sketch, chunk
is a multiple of rs
<= 8192, so it's possible for small messages to fit completely within the buffer. It also makes it easier to layer the one-shot API on top of the streaming API: the buffer size is the payload length.
A simpler design that reads and writes one chunk at a time, using an rs
-sized internal buffer:
ece_decrypt_stream_t s;
int err = ece_decrypt_stream_init(&s, ECE_SCHEME_AES128GCM, rawRecvPrivKey,
authSecret);
if (err) {
// ...
}
// Assume `ciphertext` is an encrypted block from a file, socket,
// libuv stream, etc.
ece_buf_t ciphertext;
ece_buf_t record = ciphertext;
while (record.length) {
ece_buf_t extra;
if (!ece_decrypt_stream_write(&s, &record, &extra)) {
// We don't have a complete record yet. Keep reading from the source, or wait
// for the next `uv_read_cb` call. `ece_decrypt_stream_write` will copy the
// partial record into the stream's buffer, so it's safe to free `ciphertext`
// after decryption.
break;
}
// `chunk` is an `rs`-sized slice of the stream's buffer, not a copy. This
// avoids an intermediate copy if we want to write the chunk to another
// stream, but does mean we'll need to manually copy the contents if we
// want to keep the plaintext in memory.
ece_buf_t chunk;
err = ece_decrypt_stream_read(&s, &chunk);
if (err) {
// Decryption failed. Close the source stream and free `s`.
ece_decrypt_stream_free(&s);
break;
}
// `extra` is a slice of `record` that might contain more partial or
// complete records. We'll read this record on the next turn of the loop.
record = extra;
}
ece_buf_free(&ciphertext);
ece_decrypt_stream_flush
would remain the same.
This is a good idea for a Rust port (#16), but I don't think it should be implemented in C. A streaming API would be tricky to write, tricky to audit, and more complicated to use. Plus, Web Push messages are already small, and don't need the extra buffering machinery.