test_image fails

[Atri Bhattacharya]

Description

I package Mathics for openSUSE. With release 6.0.1, when trying to run the tests during builds, we see the following issue with test_image:

[  168s] =================================== FAILURES ===================================
[  168s] ____________________ test_image[BinaryImageQ[hedy]-False-] _____________________
[  168s] 
[  168s] str_expr = 'BinaryImageQ[hedy]', str_expected = 'False', msg = '', message = ''
[  168s] 
[  168s]     @pytest.mark.skipif(
[  168s]         os.getenv("SANDBOX", False),
[  168s]         reason="Test doesn't work in a sandboxed environment with access to local files",
[  168s]     )
[  168s]     @pytest.mark.parametrize(("str_expr, str_expected, msg"), image_tests)
[  168s]     def test_image(str_expr: str, str_expected: str, msg: str, message=""):
[  168s]         result = evaluate(str_expr)
[  168s]         if result is not SymbolNull or str_expected is not None:
[  168s]             expected = evaluate(str_expected)
[  168s]             if msg:
[  168s]                 assert result == expected, msg
[  168s]             else:
[  168s] >               assert result == expected
[  168s] E               assert <Expression: System`BinaryImageQ[-Image-]> == <SymbolConstant: System`False>
[  168s] 
[  168s] test/builtin/drawing/test_image.py:69: AssertionError
[  168s] ________________ test_image[BinaryImageQ[Binarize[hedy]]-True-] ________________
[  168s] 
[  168s] str_expr = 'BinaryImageQ[Binarize[hedy]]', str_expected = 'True', msg = ''
[  168s] message = ''
[  168s] 
[  168s]     @pytest.mark.skipif(
[  168s]         os.getenv("SANDBOX", False),
[  168s]         reason="Test doesn't work in a sandboxed environment with access to local files",
[  168s]     )
[  168s]     @pytest.mark.parametrize(("str_expr, str_expected, msg"), image_tests)
[  168s]     def test_image(str_expr: str, str_expected: str, msg: str, message=""):
[  168s]         result = evaluate(str_expr)
[  168s]         if result is not SymbolNull or str_expected is not None:
[  168s]             expected = evaluate(str_expected)
[  168s]             if msg:
[  168s]                 assert result == expected, msg
[  168s]             else:
[  168s] >               assert result == expected
[  168s] E               assert <Expression: System`BinaryImageQ[-Image-]> == <SymbolConstant: System`True>
[  168s] 
[  168s] test/builtin/drawing/test_image.py:69: AssertionError
...
[  168s] =========================== short test summary info ============================
[  168s] FAILED test/builtin/drawing/test_image.py::test_image[BinaryImageQ[hedy]-False-]
[  168s] FAILED test/builtin/drawing/test_image.py::test_image[BinaryImageQ[Binarize[hedy]]-True-]

How to Reproduce

Run tests with pytest (Python 3.9/3.10).

Expected behavior

Tests completing successfully.

Your Environment

Build against the following:

• Python 3.9/3.10 (error seen with both)
• NumPy: 1.24.2
• SymPy: 1.11.1
• SciPy: 1.10.1
• Mathics-Scanner 1.13.0

In addition

Desktop (please complete the following information):

• openSUSE Tumbleweed (Linux x86_64)
• Browser: NA

Thanks in advance for any suggestions or work-arounds. Thanks also for the great software.

[yzrun]

Hi, @badshah400 , I encountered the same issue and resolved it by running pip install scikit-image. You may wan to pip install -r requirements-full.txt. I'm not sure if this applies to your situation, but I hope it helps.

[R. Bernstein]

I agree with @vitrun that this is probably the problem. In packaging, is there a package for a reduced version as well as a full version?

I would imagine that if there is just one version it would be with everything installed an thus it would have scikit-image and everything in requirements-full.txt which can be installed via pip install -e .[full]

And that brings up the question of exactly what build steps are getting used?

It is possible we may have a deficiency in how to build, or how to test in a system that doesn't have all of the full/complete set of packages, but right now to address this it would be good to have more details about the process by which this gets built.

[Atri Bhattacharya]

Thanks @vitrun and @rocky, that does seem to have been the issue.

I hit the issue while building RPM packages for openSUSE. In a nutshell, the following steps are used to run the build:

[   28s]  export USE_CYTHON=1
[   28s]  /usr/bin/python3.9 setup.py build '--executable=/usr/bin/python3.9 -s'
...
[  311s] /usr/bin/python3.9 setup.py install -O1 --skip-build --force --root /home/abuild/rpmbuild/BUILDROOT/python-Mathics-6.0.1-33.15.x86_64 --prefix /usr

Similarly with python3.10 and python 3.11.

Here is the full build log for more details:
build_log.zip

To run the tests, we use the installed Mathics and run:

pytest-3.9 --ignore=_build.python39 --ignore=_build.python310 --ignore=_build. -v

We only try to test with the absolutely necessary python modules installed, so we missed out on scikit-image (and other optional modules listed in requirements-full.txt). Would be great for our case if tests depending on optional modules were automatically skipped if the module is not found. For now, I have added them as requirements to our builds.

Full test log here:
test_log.zip

[R. Bernstein]

Ok - thanks. I'll look at and fix when I get a chance.