C2SP / wycheproof

Project Wycheproof tests crypto libraries against known attacks.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

JUnit3 runner is being used instead of JUnit4

jordanmews opened this issue · comments

The JUnit3 runner is being used instead of JUnit4.
This causes the annotations (Slow/Fast) to be ignored.
I have a fix and will submit a pull request shortly.

There was a timeout issue reported on the mailing list in Feb. It was identified then that SlowTest annotations were being ignored. I believe this was likely the cause of that issue.

Repro Steps:

  • In junit-4.12.jar!/org/junit/internal/builders/AllDefaultPossibilitiesBuilder.class, set a breakpoint in the return runner line of the runnerForClass method
  • Set a breakpoint inside of testWrappedAroundCounter (a SlowTest)
  • Run a test with debugger attached
    i.e. bazel test --java_debug --genrule_strategy=standalone --spawn_strategy=standalone BouncyCastleTest_1_57

Actual Result:

  • The breakpoint will show JUnit38ClassRunner being initialized
  • The breakpoint in testWrappedAroundCounter will be hit

Expected Result:

  • A JUnit4 runner is initialized
  • testWrappedAroundCounter does not run during a 'Fast' test

Fix:

  • Remove extends TestCase which triggers usage of the JUnit3 runner
  • Replace import static org.junit.Assert.assertArrayEquals; with import static org.junit.Assert.*;
  • Replace import junit.framework.TestCase; with import org.junit.Test;
  • Annotate all existing test methods (that are prefixed with 'test' and have no args) with @Test

Verify Fix:

  • Use same breakpoints as before in AllDefaultPossibilitiesBuilder and AesGcmTest.
  • Run the same 'Fast' test
  • The breakpoint in AllDefaultPossibilitiesBuilder.runnerForClass will show runner as BlockJunit4ClassRunner (instead of JUnit38ClassRunner)
  • The breakpoint in testWrappedAroundCounter will not be hit.

Thanks for taking time to fix this issue that I didn't know how to fix. I really appreciate it :)

Glad I could track it down. Credit also goes to the well-commented code. While stepping through, I remembered a comment specified using a 'custom Junit4 runner'.