Giters

jepsen-io / jepsen

A framework for distributed systems verification, with fault injection

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Generators are required to return a valid h/Op. A valid h/Op requires an :index. Catch 22? Or?

nurturenature opened this issue · comments

commented

Hi,

Generators are required to return a valid h/Op.

Creating an op with h/op {} requires an :index:

(assert+ (:index op) "Ops require a long :index field")

So the simplest update to existing generators will exception:

(fn [test ctx]
  ...
  (h/op {:type  :invoke
         :f     :txn
         :value [[:t id {:debit-acct  debit-acct
                         :credit-acct credit-acct
                         :amount      amount}]]}))

...

at jepsen.generator.Validate.op(generator.clj:632)

(not (history/op? op))
(conj "should be either :pending or a jepsen.history.Op")

Was hesitant to use a placeholder, e.g. -1 as it's used for unmatched ops, maybe others?
Other comments describe adding :index if it doesn't exist so didn't want to inhibit proper indexing.

The docs indicate a missing :index is Ok:

Create operations with the `op` function. Unlike most defrecords, we
  pretty-print these as if they were maps--we print a LOT of them.
    (require '[jepsen.history :as h])
    (def o (h/op {:process 0 :type :invoke, :f :read, :value [:x nil]}))
    (pprint o)
    ; {:process 0,
    ;  :type :invoke,
    ;  :f :read,
    ;  :value [:x nil],
    ;  :index nil,
    ;  :time nil}

Apologies if I've misread or missed something in the docs.

commented

I don't think you have to do anything here at all--just return a map from your fn, and it'll be automatically lifted into an Op by fill-in-op. If you want a small speed boost, then yes, use -1 in generators--the indexes are added by the interpreter later. Might push that down into generator.context later, but my sense is that almost every generator Won't Notice.

commented

(also yes, thanks, the docs are wrong as of an hour ago ;-))

commented

Confirming that sequences of:

  • maps
  • (fn [])
  • (fn [test ctx]

just Don't Notice.

But starting to use history by updating all the generators in a test to (h/op {... :index -1} throws very early, on the first op?

; map op ok, history op will Exception

(def map-op (repeat {:type :invoke :f :op  :index -1}))
(def history-op (repeat (h/op {:type :invoke :f :op  :index -1})))

(def generator
  (->> map-op
       (gen/stagger (/ 10))
       (gen/nemesis nil)
       (gen/time-limit 10)))

No implementation of method: :op of protocol: #'jepsen.generator/Generator found for class: clojure.lang.Keyword
        at clojure.core$_cache_protocol_fn.invokeStatic(core_deftype.clj:584)
        at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:576)
        at jepsen.generator$fn__9806$G__9784__9815.invoke(generator.clj:407)
        at jepsen.generator$fn__9880.invokeStatic(generator.clj:562) ...
commented

Again, I don't think you need to put :index here, and you don't need to call op yourself either. Your old code ought to work without any changes.

I think what you're hitting here miiight be that Ops aren't themselves generators the way that maps are. Might be worth adding a test for that in jepsen.generator-test, then adding an implementation to jepsen.generator--see the extend-protocol block for regular Clojure maps.

commented

Thanks for the explanations!

I took it literally that it was optimal for test generators to emit Op vs op.

But the Ops are already here and everywhere!

(->> (jepsen.store/test -1) :history first type)
jepsen.history.Op

Will convert a checker to history and post an example before closing the issue.

Ops aren't themselves generators the way that maps are.
...
see the extend-protocol block for regular Clojure maps.

That's it.
Will put together a PR with a test.

commented

Closing as Ops just shouldn't be emitted by user test generators:

; fill-in-op:

; This will be both inefficient and wrong for Ops, but users shouldn't
  ; actually be passing those to us here.
  (assert (not (instance? jepsen.history.Op op)))
commented

Yeah, might add a Generator implementation for Ops too, but it'd be a different codepath. I'm not sure if there's much point--maps are shorter, easier to write, and the cost isn't really showing up in my benchmarks any more.

commented

Just a heads-up that SNAPSHOT releases on clojars and/or in github appear to be misaligned?

It's a dynamic time :) !

Reverting back to just before original history commit is Ok.

Saw your note re using YourKit.
Breathe and depth of the current development is amazing!

Rebuilding jepsen, txn, history, elle, dom-top/core, and tesser from current github:

java.lang.NoClassDefFoundError: dom-top/core/MutableAcc-Object-Object

With clojars:

rm -rf ~/.m2/repository/jepsen/ ~/.m2/repository/io/jepsen/ ~/.m2/repository/elle/ ~/.m2/repository/dom-top/ ~/.m2/repository/tesser/

FAIL in (run!-throw-test) (interpreter_test.clj:301)
generator update throws
expected: (= (h/op {:index 0, :f :write, :value 2, :time (:time (:context e)), :process (:process (:event e)), :type :invoke}) (:event e))
  actual: (not (= {:index 0, :time 1633474, :type :invoke, :process 0, :f :write, :value 2} {:index 0, :time 1633474, :type :invoke, :process 0, :f :write, :value 2}))

FAIL in (incremental-test-write-test) (format_test.clj:376)
expected: (= 1602 size1)
  actual: (not (= 1602 1626))

FAIL in (incremental-test-write-test) (format_test.clj:377)
expected: (= 639 size2)
  actual: (not (= 639 663))

Using a previously working test:

; stats
java.lang.NoClassDefFoundError: dom-top/core/MutableAcc-long-long-long

; set-full
Assert failed:
{:type :jepsen.history/not-history,
 :history-type clojure.lang.PersistentVector}
commented

Sorry! Making changes across so many different libraries right now and things may not always be pushed to GH/Clojars at the same time. I know you're Super Eager about running on the bleeding edge so I do try to push often, but... 😅

The NoClassDefFound thing--that miiight be resolvable by just re-running. We're doing some incredibly evil dynamic classloader stuff there and it seems to break every few hundred runs for me during hot code reloading. lein clean and re-running might also help.

commented

(Oh, and I've just gone and re-pushed everything so I think you shooooould theoretically be able to clone & build tesser, dom-top, jepsen.history, jepsen.txn, elle, and jepsen via lein install and get something that works? Or hang on to a previous commit for a bit--I'm pretty darn close to buttoning this work up soon. Just a few more passes for performance on Elle and I'll be getting back to doing actual testing work and preparing a proper release.)

commented

I've just gone and re-pushed everything

Wow, thanks so much!

Just for information, an across the board pull, clean, install, still doesn't find dom-top/core/MutableAcc-Object-Object.

Or hang on to a previous commit for a bit

Will do.

Thanks for being good-natured.

commented

Huh, that is weird. Those classes are defined at macroexpansion time, so it's odd that it would ever be missing. Possibly classloader nonsense?