baidu-research / NCRF

Cancer metastasis detection with neural conditional random field (NCRF)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Pickle issue with prob_maps while generating heat maps

OpenCv30 opened this issue · comments

HI,
I am running pobs_map to generate the heatmap for 130 test images to reproduce the FROC.
I started with normal configuration --GPU 0,1 and --num_workers 5
I am seeing the following error on Windows.

Traceback (most recent call last):
for (data, x_mask, y_mask) in dataloader:
File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 501, in iter
return _DataLoaderIter(self)
File "C:\ProgramData\Miniconda3\lib\site-packages\torch\utils\data\dataloader.py", line 289, in init
w.start()
File "C:\ProgramData\Miniconda3\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
ValueError: ctypes objects containing pointers cannot be pickled
Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "C:\ProgramData\Miniconda3\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

But when I have changed the setting to - --GPU 0 and --num_workers 0 , It started running but very slow for one heatmap @level6 - it is showing 21hrs.

Can you help me on this?

commented

@OpenCv30 Sorry, but I've never tried using Windows. Can you try on using Linux machine?