many of the unit tests are failing
flodel opened this issue · comments
I am relying enough on this package (thank you so much for your work) that I would like to help. I have forked from master and was planning on working on these couple issues (#55 and #31) to eventually submit pull requests.
In the process, I wanted to run the unit tests to add my own tests, but also make sure that my
changes would not break current behaviors. Unfortunately, I see that a lot of the unit tests are failing (one example below) and I can't tell if it is because of my specific environment, or if it is indeed the case that the unit tests need some rework.
Also, I note that there are a few print() and calls to solver(..., verbose = TRUE) within the tests, which make running them in interactive mode particularly difficult to read.
I would hope that the master branch consistently pass all of its tests and create a clean output. Thank you if you can address.
Example output, after running devtools::test(filter = "g01-atoms")
Loading CVXR
Creating a generic function for ‘is.element’ from package ‘base’ in package ‘CVXR’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘CVXR’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘CVXR’
Creating a generic function for ‘%x%’ from package ‘base’ in package ‘CVXR’
Testing CVXR
✔ | OK F W S | Context
✖ | 263 2 9 | g01-atoms [3.7 s]
────────────────────────────────────────────────────────────────────────────────
test-g01-atoms.R:291: error: test the vstack function
object of type 'S4' is not subsettable
1: .handleSimpleError(function (e)
{
handled <<- TRUE
test_error <<- e
options(expressions = expressions_opt_new)
on.exit(options(expressions = expressions_opt), add = TRUE)
e$expectation_calls <- frame_calls(11, 2)
test_error <<- e
register_expectation(e)
e$handled <- TRUE
test_error <<- e
}, "object of type 'S4' is not subsettable", quote(x[i, j])) at /Users/flodel/git/CVXR0/tests/testthat/test-g01-atoms.R:291
2: eval(code, test_env)
test-g01-atoms.R:449: error: test DCP properties of partial optimize
object of type 'S4' is not subsettable
1: Problem(Maximize(square(t[1])), list(-t <= x, x <= t)) at /Users/flodel/git/CVXR0/tests/testthat/test-g01-atoms.R:449
2: .Problem(objective = objective, constraints = constraints) at /Users/flodel/git/CVXR0/R/problem.R:304
3: new(structure("Problem", package = "CVXR"), ...)
4: initialize(value, ...)
5: initialize(value, ...)
6: .local(.Object, ...)
7: Maximize(square(t[1])) at /Users/flodel/git/CVXR0/R/problem.R:314
8: .Maximize(expr = expr) at /Users/flodel/git/CVXR0/R/problem.R:73
9: new(structure("Maximize", package = "CVXR"), ...)
10: initialize(value, ...)
11: initialize(value, ...)
12: .local(.Object, ...)
13: as.Constant(expr) at /Users/flodel/git/CVXR0/R/problem.R:20
14: is(expr, "Expression") at /Users/flodel/git/CVXR0/R/constant.R:104
15: square(t[1])
16: .Square(args = list(x)) at /Users/flodel/git/CVXR0/R/elementwise.R:1100
17: new(structure("Square", package = "CVXR"), ...)
18: initialize(value, ...)
19: initialize(value, ...)
20: .local(.Object, ...)
21: lapply(args, function(arg) {
as.Constant(arg)
}) at /Users/flodel/git/CVXR0/R/atoms.R:17
test-g01-atoms.R:455: skip: test the partial_optimize eval 1-norm
Reason: empty test
test-g01-atoms.R:490: skip: test partial_optimize min 1-norm
Reason: empty test
test-g01-atoms.R:506: skip: test partial_optimize simple problem
Reason: empty test
test-g01-atoms.R:522: skip: test partial_optimize special var
Reason: empty test
test-g01-atoms.R:538: skip: test partial_optimize special constr
Reason: empty test
test-g01-atoms.R:554: skip: test partial_optimize with parameters
Reason: empty test
test-g01-atoms.R:572: skip: test partial_optimize numeric function
Reason: empty test
test-g01-atoms.R:605: skip: test partial_optimize stacked
Reason: empty test
test-g01-atoms.R:622: skip: test the NonNegative Variable class
Reason: empty test
────────────────────────────────────────────────────────────────────────────────
══ Results ═════════════════════════════════════════════════════════════════════
Duration: 3.9 s
OK: 263
Failed: 2
Warnings: 0
Skipped: 9
If it is useful, here is my sessionInfo():
R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] CVXR_0.99-6 testthat_2.2.1 devtools_1.12.0
loaded via a namespace (and not attached):
[1] Rcpp_1.0.2 xml2_1.1.1 magrittr_1.5 roxygen2_6.0.1
[5] bit_1.1-12 lattice_0.20-35 R6_2.2.2 rlang_0.4.0
[9] stringr_1.2.0 tools_3.4.1 grid_3.4.1 R.oo_1.22.0
[13] cli_1.1.0 withr_2.1.2 scs_1.2-3 commonmark_1.2
[17] assertthat_0.2.0 bit64_0.9-7 digest_0.6.12 crayon_1.3.4
[21] Matrix_1.2-10 gmp_0.5-13.1 ECOSolveR_0.5.2 codetools_0.2-15
[25] R.utils_2.9.0 memoise_1.1.0 stringi_1.1.5 compiler_3.4.1
[29] Rmpfr_0.7-1 R.methodsS3_1.7.1
The tests all pass as intended; the package would not be on CRAN if the tests on the master branch did not pass. See below.
> test_check("CVXR", filter="^g01")
══ testthat results ═══════════════════════════════════════════════════════════
[ OK: 264 | SKIPPED: 10 | WARNINGS: 0 | FAILED: 0 ]
What is true, however, is that we could clean up (e.g. skipped tests) and improve the tests to make them Rstudio-friendly and provide better messages. CVXR is a complex package and we welcome your participation, but please note that most action is happening on the 1.0 branch. This is significantly different from 0.99-x and is very actively being developed.