network partitioning
opened this issue · comments
you use time when generating the UUID.
So if the network partitions the clocks will skew.
https://en.wikipedia.org/wiki/Clock_skew
I remember readin once the google paper on this for spanner.
the buggers have atomic clocks in each data center and they account for clock skew and also bandwidth latency during clock synchronisation.
Time and Space is the biggest computing problem :)
SO its a heuristic approach. I wonder if there is benchmarking code where someone has run tests with many servers and clients and controlled the partitioning and latency to find out how useful the google approach in. If you can get ordered keys and use OT ( operational transforms ) all your chickens will come home to roost...
That's why you have machine id, process id, so clock sync between servers does not matter. Even in case of clock skew on a single server within the same process, the monotonic overflowed counter will prevent duplication.
So basically, if the same full second would happen twice, you would still have 2^24/2 unique values that could be generated in that second. Chances of collisions are very small.
Its not collisions i am worried about. i 100% agree with what you said..
Its more related to the larger issue of data consensus for distributed computing.
UUID's are super important and both papers below use clocks with UUID (various approaches). But if you resign your self to the fact that a counter cant be globally ordered and networks will partition, it seems that the only other approach is CRDT.
Google use OT and a single server according to Martin to contain the Time / Space..
Martins new paper.
https://arxiv.org/pdf/1608.03960.pdf
y.js's paper
https://github.com/y-js/yjs/files/695979/yjs.pdf
Yes, this is totally out of the scope of this library.
it indeed is. sorry will close