KB race condition

[Andrew Berezovskyi]

@likelion I have been working to remove the need to rely on Hazecast in our Twins. So far I have encountered 3 problems:

1. Synchronisation
2. Running something only once (leader election)
3. Caching

This issue is about problem no. 1. After replacing the IMap calls with the calls to the KB over SPARQL, I have fallen flat on my face encountered a new challenge with the disributed transations. In HC, I used to have only the leader do a certain operation and I have now replaced it with an if-not check, which is prone to a race condition (I recalled about the 2PC right after seeing the logs...):

scott_sandbox-twin.3@linuxkit-025000000001    | 608 [TRACE] ServiceProviderRepositoryStoreImpl - Adding a ServiceProvider
scott_sandbox-twin.3@linuxkit-025000000001    | 608 [TRACE] ServiceProviderRepositoryStoreImpl - Fetching ServiceProviders
scott_sandbox-twin.4@linuxkit-025000000001    | 5894 [TRACE] ServiceProviderRepositoryStoreImpl - Persisting the newly added ServiceProvider...
scott_sandbox-twin.4@linuxkit-025000000001    | 5909 [TRACE] ServiceProviderRepositoryStoreImpl - Adding a new ServiceProvider to the list
scott_sandbox-twin.3@linuxkit-025000000001    | 6909 [TRACE] ServiceProviderRepositoryStoreImpl - Persisting the newly added ServiceProvider...
scott_sandbox-twin.3@linuxkit-025000000001    | 6928 [TRACE] ServiceProviderRepositoryStoreImpl - Adding a new ServiceProvider to the list
scott_sandbox-twin.4@linuxkit-025000000001    | 7963 [TRACE] ServiceProviderRepositoryStoreImpl - Persisting the newly added ServiceProvider... SUCCESS!
scott_sandbox-twin.4@linuxkit-025000000001    | 8560 [INFO] RestServlet - The system is using the se.ericsson.cf.scott.sandbox.twin.servlet.Application JAX-RS application class that is named in the javax.ws.rs.Application init-param initialization parameter.
scott_sandbox-twin.3@linuxkit-025000000001    | 9655 [TRACE] ServiceProviderRepositoryStoreImpl - Persisting the newly added ServiceProvider... SUCCESS!

This bug does not manifest itself at the moment because a named graph is overwritten by an exactly the same named graph. But we need to solve it.

Do you have any suggestions?

The code is in #133, namely https://github.com/EricssonResearch/scott-eu/pull/133/files#diff-c1048e94c0d1836a46d77d3a709ac685

[Andrew Berezovskyi]

I've slept on this and figured we can implement optimistic locking on a named graph level iff SPARQL Update can perform conditional updates. But doing it properly opens more doors that it solves questions...

[Andrew Berezovskyi]

And I think I found a jackpot but it will not be super easy to implement: A Multigranurality Locking Model for RDF.

[Leonid Mokrushin]

@berezovskyi I'm not quite sure I totally understand the problem (maybe we need to talk f2f), but just wanted to mention (if that may help somehow) that semweb library provides transaction facility, so one can perform actions that involve multiple triple reads/modifications in an atomic fashion.

[Andrew Berezovskyi]

I guess you are talking about https://cliopatria.swi-prolog.org/swish/pldoc/doc/_SWI_/library/semweb/index.html ? In my case, I can't use an embedded triplestore, otherwise I would have just used https://jena.apache.org/documentation/tdb/tdb_transactions.html

The problem is that a service X is deployed as a container and scaled to N instances. Any of the N instances can write to a single KB (ie common for all instances of the service X but not globally). I need something resebling transactions at that level.

[Andrew Berezovskyi]

I have since come across A Multigranurality Locking Model for RDF and I think it's a good start in general but a bit too much for my prototype at hand.

[Andrew Berezovskyi]

cc @jadelkhoury the paper I mentioned and a description of a problem in simple terms :)

[Andrew Berezovskyi]

Workaround was applied and works OK for now. On hold till we have time to go over that paper.