Atomic operations
jcarreira opened this issue · comments
We need to provide an interface for atomic operations at the object store level for synchronization between tasks.
I propose these three operations:
- atomic set (set integer value)
- atomic increment
- atomic decrement
I think the way to go about doing this is to make these operations send (as in RDMA send) a message (in opposition to a RDMA write) and make the server handle these operations in a serial fashion.
@TylerADavis Can you look into this?
I've got an idea of how to do this on TCP as the TCP server is currently single threaded and thus serial by nature when processing (messages might not be processed in the order received necessarily due to the use of poll, but all state in the store is changed atomically), and it seems that the only requirement here would be adding three new message types, as well as code to process them.
I'm less sure of how the RDMA side of things would work, but I'd expect to be able to repurpose the fetchadd sync code. I'll look more into this.
Yes, RDMA should not be much different. It should be similar to sending an allocate command to the server.
For the RDMA server, my understanding is that the server is unaware of the ObjectID associated with any particular piece of memory. As such, should these atomic operations make use of some separate map that maps an "AtomicID" to a value?
Additionally, what should the atomic operations return? It would almost seem beneficial to have some sort of atomic fetch and an atomic exchange/test and set operation
Let's start with the TCP and leave this interface unimplemented in the RDMA side of things.
These operations can just return a boolean to indicate success (basically true most of the times). Others might be more special (test-and-set needs to return the old value).
Alright, I'll add some implementation details to the high level design doc and run it by you before I get to work implementing it
I've added an outline to the high level design doc. My basic plan is as to add the following operations, and make use of a separate key space for the atomics than the regular objects. Not all these operations will be present at the client level, as increment, decrement, and fetch sub can all be built off of fetch add.
Set
Increment
Decrement
Fetch add?
Fetch sub
Exchange
As an update to this, I've currently got it working at the network client and server levels, the next step is to set up template specialization on the store so that the methods can only be called if the store is of the atomic type.