Mutable generated values are sometimes reused while shrinking

[Stuart Cook]

Open a console at http://leebyron.com/testcheck-js/api (or equivalent). Run this program and inspect the result, and compare out.result with out.shrunk.result.

out = check(
  property(
    gen.posInt.then(x => ({ x })).then(c => gen.object({ c, unused: gen.int })).then(({ c }) => c ),
    c => {
      if (c.used) throw new Error('used');
      c.used = true;
      return (c.x === 0);
    }
  )
)

Expected behaviour: The test finds a failing case, and then shrinks it to a smaller case that finds the same failure. Thus out.result === out.shrunk.result === false.

Actual behaviour: The test finds a failing case, tries to shrink it, but reuses the existing c that has already been mutated by a previous run. This changes the result, because the precondition fails and an exception is thrown instead. Thus out.result !== out.shrunk.result.

In practice this means that once a test fails, shrinking can result in misleading error messages, because the shrink tests are run against input that has already been mutated by previous tests.