Add a backup for K/V
brandonsturgeon opened this issue · comments
Cloudflare's K/V has a major issue for Express: it doesn't guarantee that the value you store will be accessible in every region immediately.
For Express, this means the server could upload a payload and only some clients could simply not see it. Of course the issue happens in reverse, too; a client could upload a payload that the server couldn't see if they were in different regions.
K/V is super fast, faster than the alternatives, so I'd like to keep using it. I think the best plan is to also use Cloudflare's D1 when it reaches general availability as a backup.
So for every request, we'd now store the data in both KV and D1 and for every retrieval, check KV first and then check D1 if the ID wasn't found.
Or, if D1 latency gets low enough, we could just use D1 entirely 🤷
Lots of measurements and testing to do.
A few open questions:
- How does D1 latency compare to KV?
- How much extra latency is added by storing in both KV and D1?
- What does D1 pricing look like?
Oops! Looks like I was mistaken.
KV reads are cached in regional Cloudflare datacenters, meaning any update to the same key may be delayed until each datacenter expires it.
However, if a client can't find a given key in their nearest datacenter, it will reach out to the origin server to retrieve the value (and then cache it in that region's datacenter).
This means that, for us, KV is completely fine. We never update keys, so the only downside for Express would be slightly higher latencies for the first recipient in a non-origin region.
I'll still keep an eye on D1 as an option to store larger (>25mb) of data, but I'll close this issue because it's not a functional concern anymore.