test_coo_numba fails on i386 but not on amd64 arch
daissi opened this issue · comments
Describe the bug
sparse fails to build on a fresh and clean Debian/unstable i386 (but not on amd64) due to test_coo_numba tests.
dh_auto_test -O--buildsystem=pybuild
I: pybuild pybuild:284: cp -r /build/python-sparse-0.12.0+git20210614/sparse/tests /build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build
I: pybuild base:232: cd /build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build; python3.9 -m pytest
============================= test session starts ==============================
platform linux -- Python 3.9.2, pytest-6.0.2, py-1.10.0, pluggy-0.13.0
rootdir: /build/python-sparse-0.12.0+git20210614, configfile: pytest.ini
plugins: cov-2.10.1
collected 5274 items
tests/test_array_function.py ............................. [ 0%]
tests/test_compressed.py ............................................... [ 1%]
........................................................................ [ 2%]
.......XXXXXXXXxX....................................................... [ 4%]
........................................................................ [ 5%]
........................................................................ [ 6%]
.....................................................xXXXXXXXxX......... [ 8%]
........................................................................ [ 9%]
........................................................................ [ 10%]
........................................................................ [ 12%]
...........................XXxxxxXXxx................................... [ 13%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 17%]
.XXxxxxXXxx............................................................. [ 19%]
........................................................................ [ 20%]
........................................................................ [ 21%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 26%]
........................................................................ [ 27%]
............................................................... [ 28%]
tests/test_compressed_2d.py ............................................ [ 29%]
........................................................................ [ 30%]
................ [ 31%]
tests/test_coo.py ...................................................... [ 32%]
........................................................................ [ 33%]
........................................................................ [ 34%]
........................................................................ [ 36%]
........................xXXXXXXXxX...................................... [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
......................................................................XX [ 41%]
XXXXXXxX................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 45%]
............................................xXxxxxXXxx.................. [ 47%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 51%]
..................XXxxxxXXxx............................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
.................xxxX................................................... [ 60%]
........................................................................ [ 62%]
........................................................................ [ 63%]
........................................................................ [ 64%]
........................................................................ [ 66%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 75%]
.................................................................. [ 77%]
tests/test_coo_numba.py FFFF [ 77%]
tests/test_dask_interop.py . [ 77%]
tests/test_dok.py ...................................................... [ 78%]
............................................. [ 79%]
tests/test_dot.py ...................................................... [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 84%]
........................................................................ [ 85%]
............x............................................. [ 86%]
tests/test_elemwise.py ................................................. [ 87%]
........................................................................ [ 88%]
........................................................................ [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................................................ [ 99%]
..... [ 99%]
tests/test_io.py ..... [100%]
=================================== FAILURES ===================================
___________________________ TestBasic.test_roundtrip ___________________________
self = <test_coo_numba.TestBasic object at 0xed2b3e20>
def test_roundtrip(self):
c1 = sparse.COO(np.eye(3), fill_value=1)
> c2 = identity(c1)
tests/test_coo_numba.py:41:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:414: in _compile_for_args
error_rewrite(e, 'typing')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
e = TypingError('Failed in nopython mode pipeline (step: nopython frontend)\nnon-precise type pyobject\nDuring: typing of ...caused by the following argument(s):\n- argument 0: Cannot determine Numba type of <class \'sparse._coo.core.COO\'>\n')
issue_type = 'typing'
def error_rewrite(e, issue_type):
"""
Rewrite and raise Exception `e` with help supplied based on the
specified issue_type.
"""
if config.SHOW_HELP:
help_msg = errors.error_extras[issue_type]
e.patch_message('\n'.join((str(e).rstrip(), help_msg)))
if config.FULL_TRACEBACKS:
raise e
else:
> raise e.with_traceback(None)
E numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E non-precise type pyobject
E During: typing of argument at /build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/tests/test_coo_numba.py (11)
E
E File "tests/test_coo_numba.py", line 11:
E def identity(x):
E <source elided>
E """Pass an object through numba and back"""
E return x
E ^
E
E This error may have been caused by the following argument(s):
E - argument 0: Cannot determine Numba type of <class 'sparse._coo.core.COO'>
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:357: TypingError
______________________ TestBasic.test_roundtrip_constant _______________________
self = <test_coo_numba.TestBasic object at 0xeb791178>
def test_roundtrip_constant(self):
c1 = sparse.COO(np.eye(3), fill_value=1)
> c2 = identity_constant(c1)
tests/test_coo_numba.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_coo_numba.py:20: in identity_constant
return get_it()
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:414: in _compile_for_args
error_rewrite(e, 'typing')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
e = TypingError('Failed in nopython mode pipeline (step: nopython frontend)\nUntyped global name \'x\': Cannot determine N... <source elided>\n """Pass an object through numba and back as a constant"""\n return x\n ^\n')
issue_type = 'typing'
def error_rewrite(e, issue_type):
"""
Rewrite and raise Exception `e` with help supplied based on the
specified issue_type.
"""
if config.SHOW_HELP:
help_msg = errors.error_extras[issue_type]
e.patch_message('\n'.join((str(e).rstrip(), help_msg)))
if config.FULL_TRACEBACKS:
raise e
else:
> raise e.with_traceback(None)
E numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E Untyped global name 'x': Cannot determine Numba type of <class 'sparse._coo.core.COO'>
E
E File "tests/test_coo_numba.py", line 18:
E def get_it():
E <source elided>
E """Pass an object through numba and back as a constant"""
E return x
E ^
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:357: TypingError
_________________________ TestBasic.test_unpack_attrs __________________________
self = <test_coo_numba.TestBasic object at 0xeb4de7a8>
def test_unpack_attrs(self):
@numba.njit
def unpack(c):
return c.coords, c.data, c.shape, c.fill_value
c1 = sparse.COO(np.eye(3), fill_value=1)
> coords, data, shape, fill_value = unpack(c1)
tests/test_coo_numba.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:414: in _compile_for_args
error_rewrite(e, 'typing')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
e = TypingError('Failed in nopython mode pipeline (step: nopython frontend)\nnon-precise type pyobject\nDuring: typing of ...caused by the following argument(s):\n- argument 0: Cannot determine Numba type of <class \'sparse._coo.core.COO\'>\n')
issue_type = 'typing'
def error_rewrite(e, issue_type):
"""
Rewrite and raise Exception `e` with help supplied based on the
specified issue_type.
"""
if config.SHOW_HELP:
help_msg = errors.error_extras[issue_type]
e.patch_message('\n'.join((str(e).rstrip(), help_msg)))
if config.FULL_TRACEBACKS:
raise e
else:
> raise e.with_traceback(None)
E numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E non-precise type pyobject
E During: typing of argument at /build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/tests/test_coo_numba.py (54)
E
E File "tests/test_coo_numba.py", line 54:
E def unpack(c):
E return c.coords, c.data, c.shape, c.fill_value
E ^
E
E This error may have been caused by the following argument(s):
E - argument 0: Cannot determine Numba type of <class 'sparse._coo.core.COO'>
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:357: TypingError
_________________________ TestBasic.test_repack_attrs __________________________
self = <test_coo_numba.TestBasic object at 0xed1f87a8>
def test_repack_attrs(self):
@numba.njit
def pack(coords, data, shape):
return sparse.COO(coords, data, shape)
# repacking fill_value isn't possible yet
c1 = sparse.COO(np.eye(3))
> c2 = pack(c1.coords, c1.data, c1.shape)
tests/test_coo_numba.py:68:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:414: in _compile_for_args
error_rewrite(e, 'typing')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
e = TypingError('Failed in nopython mode pipeline (step: nopython frontend)\nUnknown attribute \'COO\' of type Module(<mod...line 64:\n def pack(coords, data, shape):\n return sparse.COO(coords, data, shape)\n ^\n')
issue_type = 'typing'
def error_rewrite(e, issue_type):
"""
Rewrite and raise Exception `e` with help supplied based on the
specified issue_type.
"""
if config.SHOW_HELP:
help_msg = errors.error_extras[issue_type]
e.patch_message('\n'.join((str(e).rstrip(), help_msg)))
if config.FULL_TRACEBACKS:
raise e
else:
> raise e.with_traceback(None)
E numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
E Unknown attribute 'COO' of type Module(<module 'sparse' from '/build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/sparse/__init__.py'>)
E
E File "tests/test_coo_numba.py", line 64:
E def pack(coords, data, shape):
E return sparse.COO(coords, data, shape)
E ^
E
E During: typing of get attribute at /build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/tests/test_coo_numba.py (64)
E
E File "tests/test_coo_numba.py", line 64:
E def pack(coords, data, shape):
E return sparse.COO(coords, data, shape)
E ^
/usr/lib/python3/dist-packages/numba/core/dispatcher.py:357: TypingError
=============================== warnings summary ===============================
.pybuild/cpython3_3.9_sparse/build/tests/test_coo.py: 11 warnings
/build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/sparse/_coo/core.py:1070: DeprecationWarning: resize is deprecated on all SpraseArray objects.
warnings.warn(
.pybuild/cpython3_3.9_sparse/build/tests/test_coo.py::test_html_for_size_zero
/build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/sparse/_utils.py:328: RuntimeWarning: invalid value encountered in double_scalars
density = np.float_(arr.nnz) / np.float_(arr.size)
.pybuild/cpython3_3.9_sparse/build/tests/test_coo.py::test_html_for_size_zero
/build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build/sparse/_utils.py:348: RuntimeWarning: invalid value encountered in double_scalars
np.float_(arr.nbytes)
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_coo_numba.py::TestBasic::test_roundtrip - numba.core.errors...
FAILED tests/test_coo_numba.py::TestBasic::test_roundtrip_constant - numba.co...
FAILED tests/test_coo_numba.py::TestBasic::test_unpack_attrs - numba.core.err...
FAILED tests/test_coo_numba.py::TestBasic::test_repack_attrs - numba.core.err...
===== 4 failed, 5185 passed, 35 xfailed, 50 xpassed, 13 warnings in 51.42s =====
E: pybuild pybuild:353: test: plugin distutils failed with: exit code=1: cd /build/python-sparse-0.12.0+git20210614/.pybuild/cpython3_3.9_sparse/build; python3.9 -m pytest
dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.9 returned exit code 13
To Reproduce
Trying to package last git snapshot (765e297) for Debian.
It looks similar to #393 but I don't understand why it appears only on i386 and not on amd64. Any idea what could be wrong here?
System
- OS and version: Debian unstable with python 3.9.2
sparseversion: 765e297- NumPy version: 1.19.5
- Numba version: 0.52.0
I believe there are some parts in the codebase where np.intp and np.int64 are used interchangeably, using one or the other consistently would help.
Could we offer a patch or commit to fix the issue?
For Debian
I am trying to keep one np.intp or np.int64, but there is still test_dot_type case to fail.
Please feel free to -- I'd be happy to also add a CI job to ensure compatibility. However, I lack a i386 OS and thus the ability to debug this personally.
Okay, The env I built Debian package for i386 as below, hope this helps.
# 1.
sudo apt install sbuild mmdebstrap
# 2.
sudo sbuild-createchroot --debootstrap=mmdebstrap --arch=i386 \
--include=ca-certificates \
--make-sbuild-tarball=/srv/sid-i386-sbuild.tgz \
sid /tmp/chroots/sid-i386-sbuild/ \
http://ftp.cn.debian.org/debian/
# 3
#Edit: /etc/schroot/chroot.d/sid-i386-sbuild-xx
# change the `profile` key-value to profile=default
# 4
sudo schroot -c sid-i386-sbuild You can get an i386 qemu-user chroot and I think it is easily than qemu-system