Import error from numpy compilation
PauBadiaM opened this issue · comments
Hi,
First of all I wanted to thank you for this amazing package, this implementation of Fisher's exact test is insanely fast!
I'm developing a package and I would like to include fisher
as a dependancy. The problem is, that recently (2022/01/31) my GitHub's Continuous Integration started breaking apart (I found out by updating some documentation, so not changing any dependencies), this is the error:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/__main__.py", line 18, in <module>
main(module=None)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/main.py", line 100, in __init__
self.parseArgs(argv)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/main.py", line 147, in parseArgs
self.createTests()
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/main.py", line 158, in createTests
self.test = self.testLoader.loadTestsFromNames(self.testNames,
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/loader.py", line 220, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/loader.py", line 220, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/unittest/loader.py", line 154, in loadTestsFromName
module = __import__(module_name)
File "/home/runner/work/decoupler-py/decoupler-py/tests/test_methods.py", line 2, in <module>
import decoupler as dc
File "/home/runner/work/decoupler-py/decoupler-py/decoupler/__init__.py", line 7, in <module>
from .method_ora import run_ora
File "/home/runner/work/decoupler-py/decoupler-py/decoupler/method_ora.py", line 14, in <module>
from fisher import pvalue_npy
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/fisher/__init__.py", line 3, in <module>
from .cfisher import *
File "__init__.pxd", line 242, in init cfisher
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
Error: Process completed with exit code 1.
Basically it breaks when importing fisher
. Since the packages are exactly the same version as before, I'm very puzzled on what could be happening.
Apparently is something about numpy
being compiled in different versions across packages.
The dependencies installed are:
Package Version
----------------- -------
anndata 0.7.8
bleach 3.3.0
decoupler 0.0.1
fisher 0.1.9
h5py 3.6.0
jeepney 0.6.0
joblib 1.1.0
keyring 23.0.1
llvmlite 0.37.0
natsort 8.1.0
numba 0.54.1
numpy 1.20.3
packaging 21.3
pandas 1.4.0
pip 22.0.3
pkginfo 1.7.0
pyparsing 3.0.7
python-dateutil 2.8.2
pytz 2021.3
readme-renderer 29.0
requests-toolbelt 0.9.1
rfc3986 1.4.0
scikit-learn 0.24.2
scikit-misc 0.1.3
scipy 1.8.0
SecretStorage 3.3.1
setuptools 60.7.1
six 1.16.0
sklearn 0.0
skranger 0.7.0
threadpoolctl 3.1.0
tqdm 4.62.3
webencodings 0.5.1
wheel 0.36.2
xlrd 1.2.0
I'm installing the package via pip, could this be a problem? Have you seen something similar, or could you know where the error is coming from?
Thank you for your time
Hi, I just upload v0.1.10 to pypi can you see if that resolves this?
Thanks for the quick reply @brentp!
Unfortunately the error still persists with version 1.10. What is very puzzling is that installation in several local machines (all different OS) works without a problem, but it fails when running through GitHub Actions.
I know this is out out of the scope, but am I doing something wrong here?
This is my setup.py
file:
from setuptools import setup, find_packages
setup(
name="decoupler",
version="0.0.1",
install_requires=["numba",
"sklearn",
"tqdm",
"anndata",
"skranger",
"fisher"
],
packages=["decoupler"],
python_requires=">=3.8",
classifiers=[
"Programming Language :: Python :: 3",
"Operating System :: OS Independent",
]
)
And my .yml
file
name: build
on:
push:
branches: [ main, development ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install wheel
pip install .
- name: Test with unnittest
run: |
python -m unittest tests/test_*
Maybe the error could be coming from an update in actions/setup-python@v2
or actions/checkout@v2
?
Not sure, are you using python 3.8 in your local setup?
Yes, exactly the same Python and dependencies' versions.
I have created a toy repository to reproduce the error, here:
https://github.com/PauBadiaM/test_CI_fisher
Again, in local when I run the test it works fine, but when GitHub's CI tries it, it crashes with the same error:
https://github.com/PauBadiaM/test_CI_fisher/runs/5106465412?check_suite_focus=true
I was testing and removing numba
seems to work:
https://github.com/PauBadiaM/test_CI_fisher/runs/5106492033?check_suite_focus=true
What is weird is that the numba
version is the same in my local machines than in GitHub's, so I'm at a loss of what could be happening.
must be some difference in the numpy versions. Can you check that your repo has same numpy as local?
From a scratch conda environment:
conda create -n test_CI python=3.8
conda activate test_CI
If I add numba as a requirement in setup.py
and install it with:
pip install git+https://github.com/PauBadiaM/test_CI_fisher.git
Machine | Package | Version |
---|---|---|
Local | numba | 0.55.1 |
GitHub's | numba | 0.55.1 |
Local | numpy | 1.21.5 |
GitHub's | numpy | 1.21.5 |
Local | fisher | 0.1.9 |
GitHub's | fisher | 0.1.9 |
As you can see the versions are exactly the same, however on local python -m unittest tests/test.py
works but not on GitHub's machine. Interestingly, if I bump fisher
's version to 0.1.10 it crashes in my local too.
I also tried different versions of numpy
in both local an in GitHub's machine.
With a fixed fisher==0.1.9
, versions of numpy
from 1.21.5
to 1.20.0
(included) work on local but crash in GitHub's. numpy
's version 1.19.0
crashed on local so I didn't check any further versions.