difference challenge
kostis opened this issue · comments
The difference challenge has various issues:
- If we keep it, it should become one challenge, not many (also, its first sentence currently has an off-by-one error ;-) )
- more importantly, its logic needs fixing, most likely by adding the appropriate "not"s (for example,
[10,10]is not a falsified sample for the first property because the difference is 0 and, according to the phrasing, the property succeeds in this case).
I could have of course fixed "Test 1" myself, but I am not sure which one should be kept there -- or whether the challenge should be split into three.
Edit: As a more general comment, properties that are true when a disjunction (an or) is satisfied, are not suitable as shrinking challenges, I think. To falsify them, it's enough to invalidate only one of the conjuncts that their negation contains, not both!
I agree with everything except - maybe - that a challenge must not have several parts. Why couldn't it if - as in this case - the parts are a variation of a theme?
Sure, we can have that but, for presentation purposes in the various tables and such, it may be better to have them as difference-1, difference-2, etc. (Or find some other/better way to present these multi-part challenges.)
I just hit probably the same problem.
In the first Difference problem, the smallest falsifiable example for (x,y) -> x < 10 || x == y should be (10,0), right? So either the smallest example or the property needs to change.
In second, again it's (10,0).
In third, again (10,0).
Looking at the jqwik implementation of the challenge, it indeed has a not on the right side of the ORs.
(x,y) -> x < 10 || x /= y -> (10,10)
(x,y) -> x < 10 || abs (x - y) < 1 || abs (x - y) > 4 -> (10,6)
(x,y) -> x < 10 || abs (x - y) /= 1 -> (10,9)
Looking at the jqwik implementation of the challenge, it indeed has a
noton the right side of the ORs.
(x,y) -> x < 10 || x /= y->(10,10)
(x,y) -> x < 10 || abs (x - y) < 1 || abs (x - y) > 4->(10,6)
(x,y) -> x < 10 || abs (x - y) /= 1->(10,9)
That's how I intended it to be. The text is wrong.
Why is it suggested that Test 3 is harder than Test 1, aren't they almost exactly the same?
Well, it's my bias from how jqwik does shrinking. Equality was a special case so case 1 worked from the beginning. Case 3 required special tuning. But of course, this may be completly different in other implementations.