RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.
VictorLee94 opened this issue · comments
When I'm running 2_process_patches.py, I'm getting the following runtime error. I've tried running both:
$ python code/2_process_patches.py --num_workers 1
$ python code/2_process_patches.py --num_workers 0
I solved an issue in 1_split.py by setting num_workers=0 in line 99 of compute_stats.py but when I run 2_process_patches.py with num_workers as 0, the code just doesn't move after it displays "Generating training patches".
Do you have any suggestions as to what I can do about this error?
`
+++++ Running 2_process_patches.py +++++
----- Generating training patches -----
[WindowsPath('wsi_train/0'), WindowsPath('wsi_train/1')] subfolders found from wsi_train
wsi_train\0: 3394.777501MB, 9 images, overlap_factor=1.00
wsi_train\1: 6016.922732MB, 16 images, overlap_factor=1.00
getting small crops from 9 images in wsi_train\0 with inverse overlap factor 1.00 outputting in train_folder\train\0
forrtl: error (200): program aborting due to control-C event
Image PC Routine Line Source
libifcoremd.dll 00007FFA2772DF54 Unknown Unknown Unknown
KERNELBASE.dll 00007FFA637929E3 Unknown Unknown Unknown
KERNEL32.DLL 00007FFA645F7344 Unknown Unknown Unknown
ntdll.dll 00007FFA65C426B1 Unknown Unknown Unknown
PS E:\Image Databases\Deepslide Project\deepslide> python code/2_process_patches.py --num_workers 1
############### CONFIGURATION ###############
all_wsi: all_wsi
val_wsi_per_class: 20
test_wsi_per_class: 30
keep_orig_copy: True
num_workers: 1
patch_size: 224
wsi_train: wsi_train
wsi_val: wsi_val
wsi_test: wsi_test
labels_train: labels_train.csv
labels_val: labels_val.csv
labels_test: labels_test.csv
train_folder: train_folder
patches_eval_train: patches_eval_train
patches_eval_val: patches_eval_val
patches_eval_test: patches_eval_test
num_train_per_class: 80000
type_histopath: True
purple_threshold: 100
purple_scale_size: 15
slide_overlap: 3
gen_val_patches_overlap_factor: 1.5
image_ext: jpg
by_folder: True
color_jitter_brightness: 0.5
color_jitter_contrast: 0.5
color_jitter_saturation: 0.5
color_jitter_hue: 0.2
num_epochs: 20
num_layers: 18
learning_rate: 0.001
batch_size: 16
weight_decay: 0.0001
learning_rate_decay: 0.85
resume_checkpoint: False
save_interval: 1
checkpoints_folder: checkpoints
checkpoint_file: xyz.pt
pretrain: False
log_folder: logs
auto_select: True
preds_train: preds_train
preds_val: preds_val
preds_test: preds_test
inference_train: inference_train
inference_val: inference_val
inference_test: inference_test
vis_train: vis_train
vis_val: vis_val
vis_test: vis_test
device: cpu
classes: ['0', '1']
num_classes: 2
train_patches: train_folder\train
val_patches: train_folder\val
path_mean: [0.0, 0.0, 0.0]
path_std: [0.0, 0.0, 0.0]
resume_checkpoint_path: checkpoints\xyz.pt
log_csv: logs\log_9172023_182335.csv
eval_model: checkpoints\xyz.pt
threshold_search: (0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9)
colors: ('red', 'white', 'blue', 'green', 'purple', 'orange', 'black', 'pink', 'yellow')
#####################################################
C:\Anaconda3\Lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
"class": algorithms.Blowfish,
+++++ Running 2_process_patches.py +++++
----- Generating training patches -----
[WindowsPath('wsi_train/0'), WindowsPath('wsi_train/1')] subfolders found from wsi_train
wsi_train\0: 3394.777501MB, 9 images, overlap_factor=1.00
wsi_train\1: 6016.922732MB, 16 images, overlap_factor=1.00
getting small crops from 9 images in wsi_train\0 with inverse overlap factor 1.00 outputting in train_folder\train\0
############### CONFIGURATION ###############
all_wsi: all_wsi
val_wsi_per_class: 20
test_wsi_per_class: 30
keep_orig_copy: True
num_workers: 1
patch_size: 224
wsi_train: wsi_train
wsi_val: wsi_val
wsi_test: wsi_test
labels_train: labels_train.csv
labels_val: labels_val.csv
labels_test: labels_test.csv
train_folder: train_folder
patches_eval_train: patches_eval_train
patches_eval_val: patches_eval_val
patches_eval_test: patches_eval_test
num_train_per_class: 80000
type_histopath: True
purple_threshold: 100
purple_scale_size: 15
slide_overlap: 3
gen_val_patches_overlap_factor: 1.5
image_ext: jpg
by_folder: True
color_jitter_brightness: 0.5
color_jitter_contrast: 0.5
color_jitter_saturation: 0.5
color_jitter_hue: 0.2
num_epochs: 20
num_layers: 18
learning_rate: 0.001
batch_size: 16
weight_decay: 0.0001
learning_rate_decay: 0.85
resume_checkpoint: False
save_interval: 1
checkpoints_folder: checkpoints
checkpoint_file: xyz.pt
pretrain: False
log_folder: logs
auto_select: True
preds_train: preds_train
preds_val: preds_val
preds_test: preds_test
inference_train: inference_train
inference_val: inference_val
inference_test: inference_test
vis_train: vis_train
vis_val: vis_val
vis_test: vis_test
device: cpu
classes: ['0', '1']
num_classes: 2
train_patches: train_folder\train
val_patches: train_folder\val
path_mean: [-5.959930872658708e+24, 6.978466352337589e-43, 5.739718509874451e-42]
path_std: [nan, 8.353721441928427e-22, 2.3957709225658984e-21]
resume_checkpoint_path: checkpoints\xyz.pt
log_csv: logs\log_9172023_182351.csv
eval_model: checkpoints\xyz.pt
threshold_search: (0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9)
colors: ('red', 'white', 'blue', 'green', 'purple', 'orange', 'black', 'pink', 'yellow')
#####################################################
C:\Anaconda3\Lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
"class": algorithms.Blowfish,
+++++ Running 2_process_patches.py +++++
----- Generating training patches -----
[WindowsPath('wsi_train/0'), WindowsPath('wsi_train/1')] subfolders found from wsi_train
wsi_train\0: 3394.777501MB, 9 images, overlap_factor=1.00
wsi_train\1: 6016.922732MB, 16 images, overlap_factor=1.00
getting small crops from 9 images in wsi_train\0 with inverse overlap factor 1.00 outputting in train_folder\train\0
Traceback (most recent call last):
File "", line 1, in
File "C:\Anaconda3\Lib\multiprocessing\spawn.py", line 120, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Anaconda3\Lib\multiprocessing\spawn.py", line 129, in _main
prepare(preparation_data)
File "C:\Anaconda3\Lib\multiprocessing\spawn.py", line 240, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "C:\Anaconda3\Lib\multiprocessing\spawn.py", line 291, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 291, in run_path
File "", line 98, in _run_module_code
File "", line 88, in _run_code
File "E:\Image Databases\Deepslide Project\deepslide\code\2_process_patches.py", line 13, in
gen_train_patches(input_folder=config.args.wsi_train,
File "E:\Image Databases\Deepslide Project\deepslide\code\utils_processing.py", line 144, in gen_train_patches
produce_patches(input_folder=input_subfolder,
File "E:\Image Databases\Deepslide Project\deepslide\code\utils_processing.py", line 408, in produce_patches
p.start()
File "C:\Anaconda3\Lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "C:\Anaconda3\Lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Anaconda3\Lib\multiprocessing\context.py", line 336, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "C:\Anaconda3\Lib\multiprocessing\popen_spawn_win32.py", line 45, in init
prep_data = spawn.get_preparation_data(process_obj._name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Anaconda3\Lib\multiprocessing\spawn.py", line 158, in get_preparation_data
_check_not_importing_main()
File "C:\Anaconda3\Lib\multiprocessing\spawn.py", line 138, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
`
Stale issue message