racket / rackunit

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Incorrect test counting in typed/rackunit (test count doubled)

jessealama opened this issue · comments

Working with typed/rackunit on 7.9, CS variant, macOS. Here's a rela

#lang typed/racket/base

(module+ test
  (require typed/rackunit))

(module+ test
  (test-case "one" (check-not-false 1)))

and then do raco test, you'll get

2 tests passed

If you add a test, so that we have two test-cases, like so:

#lang typed/racket/base

(module+ test
  (require typed/rackunit))

(module+ test
  (test-case "one" (check-not-false 1))
  (test-case "two" (check-not-false 2)))

you'll now get

4 tests passed

Looks like the count is always twice the number of checks. But there's more: if you make a failing test, like this:

#lang typed/racket/base

(module+ test
  (require typed/rackunit))

(module+ test
  (test-case "one" (check-not-false 1))
  (test-case "two" (check-not-false #f))) ; fails

Then you'll get:

--------------------
two
FAILURE
name:       check-not-false
location:
  /Applications/Racket v7.9/share/pkgs/rackunit-typed/rackunit/main.rkt:41:2
params:     '(#f)
--------------------
2/4 test failures

It looks like we're double counting the number of tests and double counting the number of failed tests.

The described issue doesn't arise in untyped rackunit.