Allow to define a scope/lifespan with @Provide methods
adam-waldenberg opened this issue · comments
Testing Problem
Sometimes it's useful to be able to control how providers are generating values - especially when communicating with models where it's not really appropriate to generate too many sets (like state-based models or incrementing models).
While this is already possible to some degree by taking advantage of @BeforeTry, @BeforeProperty and @BeforeContainer, giving provide methods this direct ability would add a lot of flexibility.
Suggested Solution
Add the possibility to control under what scopes values are provided by something like,
@Provide(lifespan = Lifespan.RUN)
@Provide(lifespan = Lifespan.CLASS)
@Provide(lifespan = Lifespan.PROPERTY)
@Provide(lifespan = Lifespan.TRY)
In decreasing lifespan. If a provider is invoked again within a lifespan, sets would be reused and not generated again.
And on second thought... The way values are generated at the moment I don't know if LifeSpan.TRY would make any sense here... Probably not?
Would you please outline a testing problem?
I guess https://github.com/google/guice/wiki/CustomScopes addresses a related issue, so it might be their API would work for an inspiration.
I'd subscribe to @vlsi's request for a testing scenario that would profit. There may be more elegant ways to tackle those.
As for implementation feasibility, there are a couple of challenges. One is reproducibility: Since Java has no way to ensure that an object is not mutated, object state across tries will introduce problems. Shrinking introduces even more complexity state-wise.
Not sure what kind of example to show. (Scopes in Guice is one example of this concept, Weld/CDI is another). But it makes sense for anything where the lifecycle of the object is of relevance (or even important) - or you know the set you generated already exhausts the all the possible values - depending on the model, generating it again might be very inefficient and slow down the test suite.
How to handle mutability and shrinking indeed is a good question ;) .... I doubt that's possible. This would likely have to be left up to the developer to handle correctly (i.e that they know the model they are changing the lifespan on wont be changed by some other test or by another part of the model). As a general rule of thumb, when you deal with scopes, you need a general understanding of what you are doing - this is true in most frameworks that have them. If the tester wants to trip himself up, there is a million of ways to do so.
Next implementation problem: If you keep objects around you have to introduce some lifecycle handling, e.g. for resetting the database. My potentially wrong guess is that objects you want to keep around do not profit from randomness anyway. So using a ResolveParameterHook is a feasible option.
Think of it more as objects that do benefit from randomness, but not from repeated randomness (either due to all combinations already being tried or for performance reasons).
On the JQwik side, something would need to be implemented to handle the scopes, yes... The stores that are supported by the API already do this to an extent. I'm wondering if this might be more suitable as an extension. Also wondering if this could be done via a hook. Is there currently any way to intercept or hook into @forall calls from hooks ?
@ForAll can only be intercepted by providers but a new annotation could be introduced like @Any, which would then be handled by an extension/hook. It would automatically not participate in shrinking, though.