Strict monotonicity not always held / race condition in generate-squuid*

[Ivar Refsdal]

Hi,

and thanks for a great idea and library!

I believe there is a race condition in generate-squuid* though.

Here is a way to reproduce this using CountDownLatch:

(defonce latch (atom nil))


(defn generate-squuid*
  "Return a map containing the following:
   :squuid     The v8 sequential UUID made up of a base UUID and timestamp.
   :base-uuid  The base v4 UUID that provides the lower 80 bits.
   :timestamp  The timestamp that provides the higher 48 bits.
   
   See `generate-squuid` for more details."
  []
  (let [ts (java.time.Instant/ofEpochMilli 123) ; pretend we always get the same time
        {curr-ts :timestamp} @current-time-atom]
    (if (t/before? curr-ts ts)
      ;; No timestamp clash - make new UUIDs
      (do
        (.countDown @latch)
        (.await @latch) ; wait for 2 threads to arrive here at the same time,
                        ; i.e. after the compare is done.
        (swap! current-time-atom (fn [m]
                                   (-> m
                                       (assoc :timestamp ts)
                                       (merge (u/make-squuid ts))))))
      ;; Timestamp clash - increment UUIDs
      (swap! current-time-atom (fn [m]
                                 (-> m
                                     (update :base-uuid u/inc-uuid)
                                     (update :squuid u/inc-uuid)))))))

(comment
  (do
    (reset! current-time-atom {:timestamp t/zero-time
                               :base-uuid u/zero-uuid
                               :squuid    u/zero-uuid})
    (reset! latch (java.util.concurrent.CountDownLatch. 2))
    (let [f1 (future (:squuid (generate-squuid*)))
          f2 (future (:squuid (generate-squuid*)))
          mx (last (sort [@f1 @f2]))]
         ; Now we have generated two squuids.
         ; None of them is thought to be "clashy" by the existing code.
         ; This is due to the race condition.
      (let [new (:squuid (generate-squuid*)) ; new should always be higher than the max seen so far, right?
            new-max (last (sort [mx new]))]
        (if (not= new-max new)
          (do
            (println "new:    " new)
            (println "old max:" mx)
            (println "bug"))
          (do
            (println "new:    " new)
            (println "old max:" mx)
            (println "ok this time")))))))

Evaluate that comment form a few times, and you will (sooner or later) hit the race condition. Example output:

new:     #uuid "00000000-007b-8715-82ac-485927e454a0"
old max: #uuid "00000000-007b-885f-acaa-e83df5e15fff"
bug

Thus this proves the race condition.

The correct way to handle this is to put (if (t/before? curr-ts ts) inside the function sent to swap!:

(swap! current-time-atom (fn [m]
                                   (if (t/before? (:timestamp m) ts)
                                     (-> m
                                         (assoc :timestamp ts)
                                         (merge (u/make-squuid ts)))
                                     (-> m
                                         (update :base-uuid u/inc-uuid)
                                         (update :squuid u/inc-uuid)))))

Thus I'm proposing the following definition of generate-squuid*:

(defn generate-squuid*
  "Return a map containing the following:
   :squuid     The v8 sequential UUID made up of a base UUID and timestamp.
   :base-uuid  The base v4 UUID that provides the lower 80 bits.
   :timestamp  The timestamp that provides the higher 48 bits.
   
   See `generate-squuid` for more details."
  []
  (let [ts (t/current-time)]
    (swap! current-time-atom (fn [m]
                               (if (t/before? (:timestamp m) ts)
                                 (-> m
                                     (assoc :timestamp ts)
                                     (merge (u/make-squuid ts)))
                                 (-> m
                                     (update :base-uuid u/inc-uuid)
                                     (update :squuid u/inc-uuid)))))))

I hope this makes sense.

Thanks and kind regards.

Edit: Added full proposed definition of generate-squuid*.

[Kelvin]

Thank you for the bug report! This is actually an important bug to fix, since colossal-squuid is used in applications where multi-threading and concurrency are possible (e.g. concurrent database operations). I opened a PR to fix the bug and add multi-threading tests.