Incorrect test counting in typed/rackunit (test count doubled)
jessealama opened this issue · comments
Working with typed/rackunit
on 7.9, CS variant, macOS. Here's a rela
#lang typed/racket/base
(module+ test
(require typed/rackunit))
(module+ test
(test-case "one" (check-not-false 1)))
and then do raco test
, you'll get
2 tests passed
If you add a test, so that we have two test-case
s, like so:
#lang typed/racket/base
(module+ test
(require typed/rackunit))
(module+ test
(test-case "one" (check-not-false 1))
(test-case "two" (check-not-false 2)))
you'll now get
4 tests passed
Looks like the count is always twice the number of checks. But there's more: if you make a failing test, like this:
#lang typed/racket/base
(module+ test
(require typed/rackunit))
(module+ test
(test-case "one" (check-not-false 1))
(test-case "two" (check-not-false #f))) ; fails
Then you'll get:
--------------------
two
FAILURE
name: check-not-false
location:
/Applications/Racket v7.9/share/pkgs/rackunit-typed/rackunit/main.rkt:41:2
params: '(#f)
--------------------
2/4 test failures
It looks like we're double counting the number of tests and double counting the number of failed tests.
The described issue doesn't arise in untyped rackunit
.