Arbitrary.size as an option to tweak during arbitrary construction.
SimY4 opened this issue · comments
Support for Arbitraries.size() to access the current size and <A> Arbitraries.resize(int size, Arbitrary<A> arb) for resizing the subsequent arbitrary.
Can you shortly describe what effect you'd like to get by re-sizing arbitraries/generators based on previous value?
In jqwik the size parameter is of much lower importance than for example in QuickCheck or QC-derived PBT frameworks. Many if not most generators ignore size completely. Indeed, I've been thinking about getting rid of it completely serveral times. Also, an arbitrary has no size at all, only the derived generators are initialized with a size. The default value of size is equal to the number of tries of a property, ie 1000 if you don't configure it differently.
Using Arbitrary.fixSize(..) is a shortcut for initializing any generator with the specified value.
@jlink in a library of mine for generating regex-conforming strings, I want an arbitrary to respect the size argument. But to access the size the only way for me is to implement an Arbitrary interface (here)
But this is what the jqwik doc has to say about this:
Looking at jqwik’s most prominent interfaces – Arbitrary and RandomGenerator – you might think that rolling your own implementations of these is a reasonable thing to do. I’d like to tell you that it never is, but I’ve learned that “never” is a word you should never use. There’s just too many things to consider when implementing a new type of Arbitrary to make it work smoothly with the rest of the framework.
So I'm just trying to do the right thing.
So I'm just trying to do the right thing.
Very much appreciated. Two possibilities I see:
- use
Arbitraries.fromGenerator(..) - Implement
ArbitraryDecorator
Both take care of many constraints. ArbitraryDecorator being the preferred one since fromGenerator creates an arbitrary without edge cases and without exhaustive generation.
@jlink I do use decorator for the most part. Unfortunately it and the other option still don't give access to size. Size is only surfaced in Arbitrary.generate(int) method. And you can't access it without implementing arbitrary directly.
UPD I guess resize already exist in a form of Arbitrary.fixGenSize(int) Now I only need to access size without implementing arbitrary myself. :)
Maybe the combination is worthwhile looking at. A sketch:
class SizedArbitrary implements ArbitraryDecorator<String> {
private int size;
SizedArbitrary(int size) { this.size = size; }
Arbitrary<String> arbitrary() {
return Arbitraries.fromGenerator(new MyGenerator(size));
}
}@jlink thank you, and that will essentially fix the size to some value.
It's a bit different from what I'm trying to achieve. I want the size to grow with every new iteration like you described - every new trial gives you a greater size and therefore you start by exploring smaller values slowly expanding it further. You can't achieve that via arbitrary configuration because that would be static.
It's a bit different from what I'm trying to achieve. I want the size to grow with every new iteration like you described - every new trial gives you a greater size and therefore you start by exploring smaller values slowly expanding it further. You can't achieve that via arbitrary configuration because that would be static.
I assumed that - since it's how QuickCheck's generators basically work. One of the problems is that if you did that generators would accumulate state and therefore no longer show deterministic generation behaviour when given the same random seed. QuickCheck and kin solve that by having size as an additional parameter in their next function. jqwik's design is different in order to be able to solely rely on a random seed for reproducible generation. I'm afraid there's no easy way out here.
@jlink not sure I follow you line of thought here.
I'm not suggesting you to change existing random generators. If they work fine without the size that's alright. Though built-in container like types are already utilizing genSize internally: https://github.com/jqwik-team/jqwik/blob/main/engine/src/main/java/net/jqwik/engine/properties/arbitraries/randomized/ContainerGenerator.java#L11. This is applicable to string types as well. So, why can't custom user-provided generators do that?
To modify your example a little. I want my custom generator to be defined like this:
Arbitraries.size().flatMap(size -> Arbitraries.fromGenerator(new MyGenerator(size)))
// or maybe
Arbitraries.fromGenerator(size -> new MyGenerator(size)))To me, size is essentially an Arbitrary.pure(int) value that only differs in between trial runs. But between runs your starting seed is different anyway because otherwise it's not gonna vary much in between runs that's why I'm having trouble to understand your point about deterministic runs with sizes. If your seed is different what could be the issue of making size one more variable option available for Arbitrary building?
@jlink not sure I follow you line of thought here.
I’ve probably rambled too much in the theory of value generation and looked too little into what you want to achieve. :-/
I'm not suggesting you to change existing random generators. … So, why can't custom user-provided generators do that?
Sure they can. My point: In existing generators size is being used to change the distribution of values, eg larger size leads to larger collections on average. What generators should not do is to generate different values depending on what they generated before. Changing size from one next() call to the subsequent one would do exactly that.
To modify your example a little. I want my custom generator to be defined like this:
Arbitraries.size().flatMap(size -> Arbitraries.fromGenerator(new MyGenerator(size))) // or maybe Arbitraries.fromGenerator(size -> new MyGenerator(size)))
That seems to make a lot of sense and I’m wondering why I left out the size parameter in the first place. Let me think about that…
@SimY4 Had another look at it and cannot see any valid argument against Arbitraries.fromGenerator(Function<Int, RandomGenerator>). Would that - together with the suggestions from above - fulfill your needs? Then I'd put it on the list for 1.7.5.
Starting work on this feature in API.EXPERIMENTAL status.
First obstacle: The method cannot be called fromGenerator since due to generics parameter cancelling
Arbitraries.fromGenerator(RandomGenerator) looks like an override of Arbitraries.fromGenerator(Function<Integer, RandomGenerator>) with more specific parameter to the compiler. This is an incompatible change.
Using supplyGenerator(Function<Integer, RandomGenerator>) for now. Any other suggestions?
@jlink I was actually thinking of proposing arbitrary(IntFunction<RandomGenerator> arbitrary) because it's essentially what arbitrary interface is without edge cases and other combinators.
Seems like a very generic name at first glance. Have to think about that.
@SimY4 Having slept over it, Arbitraries.arbitrary(..) is just too generic considering the rest of jqwik's API.
This name suggests that it's THE default to choose if you don't know what else to do.
Names I've considered:
supplyGenerator, createGenerator, fromSuppliedGenerator, fromGeneratorUsingSize
More ideas? Favourites?
@jlink I like the inclusion of the word size in the name since using a generic functional interface there won't actually help to realize that int argument is size.
I would vote for the last option of yours.
Some other options I have are: fromSizedGenerator, fromGeneratorSized, sizedGenerator
Adding a postfix to the original method seems reasonable. Three options remaining:
fromGeneratorSizedfromGeneratorUsingSizefromGeneratorWithSize
I have decided on fromGeneratorWithSize since withSize is already widely used in other contexts.
Available in 1.8.0-SNAPSHOT.