joblib / joblib

Computing with Python functions.

Home Page:http://joblib.readthedocs.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

"ValueError: Stop argument for islice() must be None or an integer: 0 <= x <= sys.maxsize."

jnc-nj opened this issue · comments

def restore_func(images, restorer):
    for idx in range(len(images)):
        img = cv2.cvtColor(_img, cv2.COLOR_RGB2BGR)
        # restore faces and background if necessary
        cropped_faces, restored_faces, r_img = restorer.enhance(
            img,
            has_aligned=False,
            only_center_face=False,
            paste_back=True)
        r_img = cv2.cvtColor(r_img, cv2.COLOR_BGR2RGB)
        yield r_img

def accum_func(r_img):
    return r_img

job = Parallel(n_jobs=batch_size/2, return_as='generator', pre_dispatch='1.5*n_jobs', verbose=10)(delayed(accum_func)(r_img) for r_img in restore_func(images, restorer))`

I run into the following error for the above code:

File "/workspace/SadTalker/src/utils/face_enhancer.py", line 41, in enhancer_list
   gen = enhancer_generator_no_len(images, method=method, bg_upsampler=bg_upsampler, batch_size=batch_size)
 File "/workspace/SadTalker/src/utils/face_enhancer.py", line 129, in enhancer_generator_no_len
   job = Parallel(n_jobs=batch_size/2, return_as='generator', pre_dispatch='1.5*n_jobs', verbose=10)(delayed(accum_func)(r_img) for r_img in restore_func(images, restorer))
 File "/usr/local/lib/python3.8/dist-packages/joblib/parallel.py", line 1942, in __call__
   next(output)
 File "/usr/local/lib/python3.8/dist-packages/joblib/parallel.py", line 1580, in _get_outputs
   self._start(iterator, pre_dispatch)
 File "/usr/local/lib/python3.8/dist-packages/joblib/parallel.py", line 1563, in _start
   if self.dispatch_one_batch(iterator):
 File "/usr/local/lib/python3.8/dist-packages/joblib/parallel.py", line 1426, in dispatch_one_batch
   islice = list(itertools.islice(iterator, big_batch_size))
ValueError: Stop argument for islice() must be None or an integer: 0 <= x <= sys.maxsize.

The base examples here work fine, which is quite confusing considering i'm following the sample code here

Any advice would be appreciated. Thanks in advance!

solved, silent failure when gpu ooms.
reducing batch_size fixes this.

Thanks for the report. I think this is related to the behavior reported in #1489? The error reporting should be improved with #1491 hopefully.