Get-or-insert-with semantics / Dogpile Effect Mitigation

[Zoey]

Async Caches benefit from being able to ensure that - if a value is not currently cached - a future imminently fulfilling the entry will be passed to the other asynchronous readers. I haven't found a way to implement this pattern without adding another layer of internal mutability within the existing Stretto cache; is there an officially endorsed way to deduplicate cache fulfillment requests? If not, I'd like to request the feature.

[Al Liu]

Hi Dessix, thank you suggestion. Currently, stretto does not provide a function get_or_insert like Hashmap or etc. I will add something like this in next release. Besides, would you mind to provide a mock example for your use cases for this feature request?

[Zoey]

The important part is that it only calls the factory function to create the value when the value is missing- but it also prevents any subsequent calls from beginning construction while it is in the process of fulfilling the first.

A more concrete example is if I have an LRU cache to keys on a remote key/value store, and want to ensure that I only query the remote store once if my cache is missing the value- even if I suddenly receive many requests for it at once. Once it's fulfilled, all active requesting tasks can share the result.

c.get_or_try_fulfill_with(|k: &Key| async move {
  // some expensive, async operation to retrieve a value
  let response = myExpensiveRequest.await?;
  Ok(response)
})