Guarantee get after on_miss?
kilobyte opened this issue · comments
There are two recommended ways to use the cache, one of them being supplying an on_miss callback. Alas, if there's a lot of puts going on (likely if there's a "preload" operation going on a live server) or the callback is very slow after put (writes a log/etc which stalls), it's possible for the entry to have already been evicted before get gets around to retry the request.
And, knowing people, no one is going to check for that. A rare unexpected failure will crash the program.
One way to fix this would be a TLS variable that says we're in an on_miss callback. The put done by that thread would have its refcount bumped, decreased only once the get completes (there might be more than one put, but only one successful for that specific key).
Thoughts...?
A better idea: instead of complex mucking with refcount, we can put into that TLS variable arguments for the pending get (key, ksize, vbuf, vbufsize, offset, &vsize).
Then, a put, before even checking total cache size checks if there's a pending get — if yes and the key matches, we can satisfy it immediately, faster (DRAM->DRAM copy, no fragmentation). So even if the put fails for whatever reason, or the entry gets evicted before we return, all is done. There's also no need for the callback to tell us whether it succeeded as we already know that.