microsoft / FASTER

Fast persistent recoverable log and key-value store + cache, in C# and C++.

Home Page:https://aka.ms/FASTER

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

FasterKV: Is there a way to wait for flush globally including all pending operations?

AqlaSolutions opened this issue · comments

At some point we need to ensure that everything we've added to FasterKV so far is persisted including all Upsert calls issued previously. We have many FasterKV parallel sessions which do Upserts. Is there a way to globally wait for all pending IO to complete before initiating a checkpoint?

We don't want to separately call CompletePending on each session, there are too many of them and it would be a huge synchronization overhead (because we use actor model and each actor has its own session).

We don't have this option. It might be better to keep a shared pool of sessions, borrow from the pool, and return to the pool within each actor grain. See this for example:

https://github.com/microsoft/FASTER/blob/main/cs/playground/AsyncStress/SerializedFasterWrapper.cs#L16