joblib / joblib

Computing with Python functions.

Home Page:http://joblib.readthedocs.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

joblib memmap issue

TimothyADavis opened this issue · comments

Dear joblib team,

Thanks for all your hard work on this great library!

Since upgrading from v1.1.0 to v1.3.x I've been having a problem with joblib, that turns out to be related to the memmap functionality. If I run as standard I get the following error - which seems to relate to memmap.filename returning None (or not being defined). Setting max_nbytes=None seems to fix things. I am on Python 3.9.16, numpy 1.25.2, and this happened with both joblib 1.3.1 and 1.3.2 (but not 1.1.0).

joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/anaconda3/lib/python3.9/site-packages/joblib/externals/loky/process_executor.py", line 426, in _process_worker
call_item = call_queue.get(block=True, timeout=timeout)
File "/opt/anaconda3/lib/python3.9/multiprocessing/queues.py", line 122, in get
return _ForkingPickler.loads(res)
File "/opt/anaconda3/lib/python3.9/site-packages/joblib/numpy_pickle.py", line 600, in load_temporary_memmap
add_maybe_unlink_finalizer(obj)
File "/opt/anaconda3/lib/python3.9/site-packages/joblib/_memmapping_reducer.py", line 72, in add_maybe_unlink_finalizer
"".format(type(memmap), id(memmap), os.path.basename(memmap.filename),
File "/opt/anaconda3/lib/python3.9/posixpath.py", line 142, in basename
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType
"""

Many thanks.