unmaintained dependency `matrixprofile` makes `tsfresh` uninstallable on python 3.10
mmp3 opened this issue · comments
The problem:
tsfresh cannot be installed on python 3.10 because it has matrixprofile as a dependency, yet matrixprofile cannot be installed on python 3.10 and is no longer maintained.
Anything else we need to know?:
matrixprofile is superseded by stumpy.
Environment:
- Python version: 3.10
Let's help @kempa-liehr fix this.
SOLUTIONS:
- Make matrixprofile package's installation optional using extras_require in the setup.py
- Implement stumpy instead of matrixprofile.
Anyone familiar on how to use extras_require in the setup.py to make matrixprofile optional? Anyone able to post a code that would replace matrixprofile functions on tsfresh? I will have a go and post it here
@erezinman @aiwalter @PJPRoche @williamgilpin @enrique-wesper
Making matrixprofile package optional:
- Changes on: setup.py
For making matrixprofile optional, the setup.py solution is like this:
import sys
from setuptools import find_packages, setup
with open("README.md") as f:
long_description = f.read()
with open("requirements.txt") as f:
requirements = [line for line in f if not line.startswith("#")]
needs_sphinx = {"build_sphinx", "upload_docs"}.intersection(sys.argv)
sphinx = ["sphinx", "sphinx_rtd_theme"] if needs_sphinx else []
simple_requirements = [r for r in requirements if r != "matrixprofile>=1.1.10,<2.0.0"]
advanced_requirements = [r for r in requirements if r == "matrixprofile>=1.1.10,<2.0.0"]
setup(
use_scm_version=True,
long_description=long_description,
long_description_content_type="text/markdown",
setup_requires=["setuptools_scm"] + sphinx,
packages=find_packages(exclude=["tests.*", "tests"]),
extras_require={
"advanced": advanced_requirements, # Advanced features
"simple": simple_requirements, # Basic features
}
)
- Changes on: feature_extraction/feature_calculators.py
Note: "matrix_profile" is a tsfresh function, while "matrixprofile" is the outdated package in question.
The function "matrix_profile" from feature_calculators.py script will still try to import it. The solution is to check if matrixprofile package is installed before importing it. I suggest making the following changes.
import importlib.util
package = "matrixprofile"
is_present = importlib.util.find_spec(package)
if is_present is not None:
import matrixprofile as mp
And
if is_present is not None:
@set_property("fctype", "combiner")
def matrix_profile(x, param):
"""
Calculates the 1-D Matrix Profile[1] and returns...
"""
[...]
return [(key, value) for key, value in res.items()]
- Changes on: across tsfresh package
After the implementation on feature_calculators.py, the only thing needed would be to place a if is_present is not None: on where matrix_profile is called across tsfresh package.
Calling importlib.util.find_spec(package) is fast but you could make it faster by simply using from ...feature_extraction.feature_calculators import is_present, this will help scale on big data. and will call importlib.util.find_spec(package) only once.
I hope this answer is clear enough to give a sense of how to fix this. If it is too convoluted or it could be improved please point it out.
@kempa-liehr @erezinman @aiwalter @PJPRoche @williamgilpin @enrique-wesper @mmp3 @bvanelli
References:
- https://stackoverflow.com/questions/62713525/declare-optional-dependencies-within-extras-require
- https://www.codespeedy.com/how-to-check-if-a-python-package-is-installed/#:~:text=Using%20the%20import%20keyword%20in,to%20do%20the%20same%20thing.
- https://stackoverflow.com/questions/19289171/importing-a-variable-from-one-python-script-to-another
@arturdaraujo Just separate the matrixprofile requirement from the rest. I'll try creating a PR if I'll get to it later. Otherwise, good job!
Also, you could simply import the package normally, but wrap this with try: ... except ImportError: ... without using importlib. Again, I'll try to do it later on.
I also think try: ... except ImportError: ... is a much better solution now that you mentioned it, no need for using importlib.
Thanks for the reply
I just realized that there is a PR #985
Can you guys also check if that's adequate? It seems pretty good to me
@kempa-liehr @erezinman @aiwalter @PJPRoche @williamgilpin @enrique-wesper @mmp3 @bvanelli @michaelosthege
PR has been merged and a new version 0.20.0 released :)
(if you still see issues, please let us know by!)
Thank you for your time and effort!
@nils-braun great! should we also update it here?
Lines 10 to 30 in d059eec
Thanks @aiwalter ! I have added py 3.10 as a classifier. I have not added an upper bound to the python_requires as the ´tsfresh´ package itself has no limitation on it (some dependencies might have though).
Also, I have not added a 3.11 classifier as currently the dependencies will not allow tsfresh to be installed on py 3.11.
@nils-braun I still cannot use tsfresh with python 3.10.9. I am getting the following error:
ModuleNotFoundError: TSFreshFeatureExtractor requires python version to be <3.10, but system python version is 3.10.9 (v3.10.9:1dd9be6584, Dec 6 2022, 14:37:36) [Clang 13.0.0 (clang-1300.0.29.30)].
I am using the main branch with latest commit 2e49614
Indeed I was importing from sktime
from sktime.transformations.panel.tsfresh import TSFreshFeatureExtractor