JUnit 5 + PreparedDbExtension reuses the same database for multiple tests
MaxAller opened this issue · comments
For legacy JUnit integration, the documentation says for preparedDatabase:
The rule contributes Config switches to configure each test case to get its own database.
and then the PreparedDbRule implementation overrides the before callback, which as we know runs before every test case.
The JUnit 5 integration documentation makes the same claim, but accidentally overrides beforeAll (which runs before ALL tests in a class) instead of beforeEach.
Somewhat related to #138 but a different class.
So you would prefer the JUnit 5 Extensions to implement BeforeEachCallback instead of BeforeAllCallback? Or would you rather see the Javadoc adjusted? Or would you prefer both alternatives?
I get that there might be some cases where the beforeEach might be preferable -- but I am currently using a custom Extension (that uses beforeAll) for one of my projects because it doubles the speed of my integration tests (I do have to use a DELETE FROM … without WHERE in an @AfterEach method, though).
I'd prefer the junit 4 and 5 implementations be consistent with each other and for the documentation to be correct 😃
But yeah, having the choice of beforeEach or beforeAll would be nice. I agree that the tests are a lot slower if you nuke the DB between each test case (~200ms for me, IIRC), but also more correct; I really don't want test cases to see data from other test cases. If you think of tests involving the DB as integration tests that shouldn't always be run, the slowness is a bit easier to rationalize.
Perhaps the rule can override both each and all callbacks, and based on a parameter can perform setup/cleanup in one or the other?
I'd prefer the junit 4 and 5 implementations be consistent with each other and for the documentation to be correct 😃
Yeah, consistency is a good point 😃
It's been a while since I used JUnit 4 — was it even possible to define a Rule for @BeforeClass?
But yeah, having the choice of beforeEach or beforeAll would be nice. I agree that the tests are a lot slower if you nuke the DB between each test case (~200ms for me, IIRC), but also more correct; I really don't want test cases to see data from other test cases. If you think of tests involving the DB as integration tests that shouldn't always be run, the slowness is a bit easier to rationalize.
That's why I clean up the data after each test case with @AfterEach. The DB is back to its initial state and there are no side effects and no shared data between the test cases. I agree that integration tests may be slower and I'm not sure how much longer the tests cases took, but even ~200ms is more than enough if you have 10 or more test cases in a class — at least if you are as impatient as I am 😅
Perhaps the rule can override both each and all callbacks, and based on a parameter can perform setup/cleanup in one or the other?
I don't think that's possible. Neither @RegisterExtension nor @ExtendWith offer parameters to be passed to the Extension, so how should it be configured?
I think the best choice would be to provide two Extensions (maybe extending the same abstract class?), one that works per test case with beforeEach and one that works per class with beforeAll. Honestly, I thought that _SingleInstance_PostgresExtension was intended to cover the latter.
I don't think that's possible.
I mean this method could accept an optional second parameter (enum) that specified the mode. Doesn't need to be on the annotation.
Yes, you're absolutely right. I was a bit fixated on the declarative registration of extensions. I could update my PR #138 tomorrow or the day after.
The Javadoc might need some enhancements as well.
FWIW @qutax we merged your PR into 1.0.0 and did run into issues with some apps. We created a LegacySingleInstancePostgres with the old code, which DID function correctly.
The symptoms were basically it seemed to fire in the wrong order and not set up PG in time for a BeforeEach to work properly.
@mikebell90 is it possible that the extension isn't registered at class level in these cases?
See also: https://github.com/junit-team/junit5/blob/f58cd419755846f1476e8d15783438de8d7aede4/junit-jupiter-api/src/main/java/org/junit/jupiter/api/extension/BeforeAllCallback.java#L24-L25
btw: did you take @MaxAller's comment into consideration? I have to admit that I did not update my PR "tomorrow or the day after" almost two years ago 🙈
The class utililizing the provider in question has esoteric needs, and is probably not a great example. In brief Yes, if you make the field static and have the RegisterExtension it works dandy, and that covers 99% of the matter. For this class, it needed to override in the constructor (which I think is a violation of sanity), specifically because it needs to copy some files over to the container via bind mount and must do a customize on the builder.
So to be fair it's a weird case, probably should be replaced by EmbeddedPostgres directly in the constructor and some cleanup method. Since I was trying to get 1.0.0 out, I "cheated" and committed the Legacy version in to look at it later, and give users a temporary hatch.
So yeah you are probably right in normal usage it works fine.
No never noticed that particular user's comment tbh. I guess I'd probably prefer to keep the classes separate and use common code, rather than blur concerns and double implement (which I think is what he was asking for)