jlink / shrinking-challenge

Comparing shrinking approaches and performance across different PBT libraries

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Coupling Phrasing

kostis opened this issue · comments

The phrasing of the coupling challenge is currently vague and needs to be improved.

In particular, it needs to say that it involves a list of (arbitrary sized / certain range only?) integers and mention explicitly what is the "unusual way" that integers are coupled to their position.

What about that:

"Given a list of integers (<= 10) in which all values are smaller than the size of the list, the property will fail:

  • if an element is not the same as its index in the list (zero-based indexing)
  • and if this element used as an index points to an element that - when its value is also used as an index - points back"

At least that's my prose interpretation of the original Python code:

@eval_given(st.lists(st.integers(0, 10)))
    def test(ls):
        assume(all(v < len(ls) for v in ls))
        for i, j in enumerate(ls):
            if i != j:
                assert ls[j] != i

One could also (maybe more clearly) state it as a property of a pair of elements in the list.

There's also a generation challenge to come up with a random failing sample but I guess that constraining the integers to max 10 helps greatly here.

When I first read the above responses, my first reaction was to reply that "there are infinitely many integers (<= 10) that when you put them a list their values are smaller than the length of the list" :-) so the specification of the challenge still needs some small improvement (namely that the integers should be in the range [0,10] -- or actually in the range [0,min(10,length(L))] if I understand the problem statement correctly).

However, after some thought, I now feel that this (artificial) constraint on the range of the integers makes this problem very simple, both as a shrinking and as a generation challenge (there are very few inputs in the first place, and very few of them make the property true).

My vote is to drop this problem -- we can find better challenges.

Do you have something in mind to replace it?

For me the essence to keep is the coupling between all possible lists and those that fulfill the initial assumption. Some PBT tools have a hard time to keep up the assumption and still shrink effectively.