The library dose not work with the latest allennlp
apohllo opened this issue · comments
Contrary to the instructions, I installed the latest allennlp library. Unfortunately I got the following error:
Traceback (most recent call last):
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 918, in save_global
obj2, parent = _getattribute(module, name)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 266, in _getattribute
.format(name, obj))
AttributeError: Can't get local attribute 'wrap_function.<locals>.WrappedFunc' on <function wrap_function at 0x2b6ace9a7d90>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 639, in save_global
return Pickler.save_global(self, obj, name=name)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 922, in save_global
(obj, module_name, name))
_pickle.PicklingError: Can't pickle <class 'ray.tune.trainable.wrap_function.<locals>.WrappedFunc'>: it's not found as ray.tune.trainable.wrap_function.<locals>.WrappedFunc
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/net/people/plgapohl/python-albert-pytorch/bin/allentune", line 11, in <module>
load_entry_point('allentune', 'console_scripts', 'allentune')()
File "/net/people/plgapohl/allentune/allentune/commands/__init__.py", line 67, in main
args.func(args)
File "/net/people/plgapohl/allentune/allentune/commands/search.py", line 126, in search_from_args
executor.run(args)
File "/net/people/plgapohl/allentune/allentune/modules/ray_executor.py", line 94, in run
self.run_distributed(run_func, args)
File "/net/people/plgapohl/allentune/allentune/modules/ray_executor.py", line 58, in run_distributed
register_trainable("run", run_func)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/tune/registry.py", line 49, in register_trainable
_global_registry.register(TRAINABLE_CLASS, name, trainable)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/tune/registry.py", line 88, in register
self._to_flush[(category, key)] = pickle.dumps(value)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 881, in dumps
cp.dump(obj)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 268, in dump
return Pickler.dump(self, obj)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 409, in dump
self.save(obj)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 648, in save_global
return self.save_dynamic_class(obj)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 495, in save_dynamic_class
save(clsdict)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v) [26/1833]
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 410, in save_function
self.save_function_tuple(obj)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 553, in save_function_tuple
save(state)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 781, in save_list
self._batch_appends(obj)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 808, in _batch_appends
save(tmp[0])
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 410, in save_function
self.save_function_tuple(obj)
File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 553, in save_function_tuple
save(state)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 521, in save
self.save_reduce(obj=obj, *rv)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 634, in save_reduce
save(state)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 521, in save
self.save_reduce(obj=obj, *rv)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 634, in save_reduce
save(state)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 781, in save_list
self._batch_appends(obj)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 808, in _batch_appends
save(tmp[0])
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 521, in save
self.save_reduce(obj=obj, *rv)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 634, in save_reduce
save(state)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 496, in save
rv = reduce(self.proto)
TypeError: can't pickle _thread.RLock objects
Any idea how I can fix that?
To my complete confusion, this is the last version that still works:
- allenai/allennlp@11f4307
and this is the first version that doesn't: - allenai/allennlp@26e313b
The only difference is the bump in version from rc3 to rc4.
I changed the testing procedure (I installing from local git repot now) and found out, that the 26e313b version still, works, but the next version doesn't:
allenai/allennlp@4a6023b
I will investigate the differences between these version.
Well, reverting that commit (4a6023b) makes the more recent version (592c65341891ccadec31a624c4e6f062e8455723 from 13th may) of allennlp work properly with allentune. It seems that the problem lays in that commit. Still, I have no idea why changing the logging infrastructure causes the error.
What latest version of allennlp is compatible with allentune? I'm talking about official releases of allennlp, i.e. not a source installation.
@ofersabo I haven't tried it but I suppose it should work with either v0.8.5 or v0.9.0 since these are the commits closest to the commit mentioned in the documentation.
@apohllo on my end the installation isn't functional even in 0.9.0 and. 0.8.5 versions.
I get these error messages:
============================= test session starts ==============================
platform linux -- Python 3.6.10, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- /home/nlp//anaconda3/envs/tune/bin/python
cachedir: .pytest_cache
rootdir: /home/nlp//translation/mt_ex_2/allentune, inifile: pytest.ini
plugins: flaky-3.6.1, cov-2.10.0
collected 4 items / 1 error / 3 selected
==================================== ERRORS ====================================
__________________ ERROR collecting tests/test_example_run.py __________________
ImportError while importing test module '/home/nlp//translation/mt_ex_2/allentune/tests/test_example_run.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_example_run.py:1: in
from allentune.modules import AllenNlpRunner, RayExecutor
allentune/modules/init.py:1: in
from allentune.modules.allennlp_runner import AllenNlpRunner
allentune/modules/allennlp_runner.py:14: in
from allennlp.common.util import import_module_and_submodules
E ImportError: cannot import name 'import_module_and_submodules'
=============================== warnings summary ===============================
/home/nlp//anaconda3/envs/tune/lib/python3.6/site-packages/plac_ext.py:6
/home/nlp/____/anaconda3/envs/tune/lib/python3.6/site-packages/plac_ext.py:6: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================== short test summary info ============================
ERROR tests/test_example_run.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 5.73s ==========================
When I'm trying to run a test directly I get this error:
from allennlp.common.util import import_module_and_submodules
ImportError: cannot import name 'import_module_and_submodules'
these errors are in allennlp==0.9.0
I encountered the same error. As far as I remember, commenting out the line with the import fixed it.
OK thanks, so I have to install from source...
Hi all, sorry about the delay on this. This should be fixed by #12. Feel free to re-open if you're still having issues -- make sure you use allennlp 1.0.