train.py error

[123liluky]

After installing torch0.4 in python3.6, I run the train.py. At the beginning, error was

So, I add interplolate copy from net like below into functional.py

def interpolate(input, size=None, scale_factor=None, mode='nearest', align_corners=None):
r"""Down/up samples the input to either the given :attr:size or the given
:attr:scale_factor
The algorithm used for interpolation is determined by :attr:mode.
Currently temporal, spatial and volumetric sampling are supported, i.e.
expected inputs are 3-D, 4-D or 5-D in shape.
The input dimensions are interpreted in the form:
mini-batch x channels x [optional depth] x [optional height] x width.
The modes available for resizing are: nearest, linear (3D-only),
bilinear (4D-only), trilinear (5D-only), area
Args:
input (Tensor): the input tensor
size (int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int]):
output spatial size.
scale_factor (float or Tuple[float]): multiplier for spatial size. Has to match input size if it is a tuple.
mode (string): algorithm used for upsampling:
'nearest' | 'linear' | 'bilinear' | 'trilinear' | 'area'. Default: 'nearest'
align_corners (bool, optional): if True, the corner pixels of the input
and output tensors are aligned, and thus preserving the values at
those pixels. This only has effect when :attr:mode is linear,
bilinear, or trilinear. Default: False
.. warning::
With align_corners = True, the linearly interpolating modes
(linear, bilinear, and trilinear) don't proportionally align the
output and input pixels, and thus the output values can depend on the
input size. This was the default behavior for these modes up to version
0.3.1. Since then, the default behavior is align_corners = False.
See :class:~torch.nn.Upsample for concrete examples on how this
affects the outputs.
"""
from numbers import Integral
from .modules.utils import _ntuple
def _check_size_scale_factor(dim):
if size is None and scale_factor is None:
raise ValueError('either size or scale_factor should be defined')
if size is not None and scale_factor is not None:
raise ValueError('only one of size or scale_factor should be defined')
if scale_factor is not None and isinstance(scale_factor, tuple)
and len(scale_factor) != dim:
raise ValueError('scale_factor shape must match input shape. '
'Input is {}D, scale_factor size is {}'.format(dim, len(scale_factor)))
def _output_size(dim):
_check_size_scale_factor(dim)
if size is not None:
return size
scale_factors = _ntuple(dim)(scale_factor)
# math.floor might return float in py2.7
return [int(math.floor(input.size(i + 2) * scale_factors[i])) for i in range(dim)]
if mode in ('nearest', 'area'):
if align_corners is not None:
raise ValueError("align_corners option can only be set with the "
"interpolating modes: linear | bilinear | trilinear")
else:
if align_corners is None:
warnings.warn("Default upsampling behavior when mode={} is changed "
"to align_corners=False since 0.4.0. Please specify "
"align_corners=True if the old behavior is desired. "
"See the documentation of nn.Upsample for details.".format(mode))
align_corners = False
if input.dim() == 3 and mode == 'nearest':
return torch._C._nn.upsample_nearest1d(input, _output_size(1))
elif input.dim() == 4 and mode == 'nearest':
return torch._C._nn.upsample_nearest2d(input, _output_size(2))
elif input.dim() == 5 and mode == 'nearest':
return torch._C._nn.upsample_nearest3d(input, _output_size(3))
elif input.dim() == 3 and mode == 'area':
return adaptive_avg_pool1d(input, _output_size(1))
elif input.dim() == 4 and mode == 'area':
return adaptive_avg_pool2d(input, _output_size(2))
elif input.dim() == 5 and mode == 'area':
return adaptive_avg_pool3d(input, _output_size(3))
elif input.dim() == 3 and mode == 'linear':
return torch._C._nn.upsample_linear1d(input, _output_size(1), align_corners)
elif input.dim() == 3 and mode == 'bilinear':
raise NotImplementedError("Got 3D input, but bilinear mode needs 4D input")
elif input.dim() == 3 and mode == 'trilinear':
raise NotImplementedError("Got 3D input, but trilinear mode needs 5D input")
elif input.dim() == 4 and mode == 'linear':
raise NotImplementedError("Got 4D input, but linear mode needs 3D input")
elif input.dim() == 4 and mode == 'bilinear':
return torch._C._nn.upsample_bilinear2d(input, _output_size(2), align_corners)
elif input.dim() == 4 and mode == 'trilinear':
raise NotImplementedError("Got 4D input, but trilinear mode needs 5D input")
elif input.dim() == 5 and mode == 'linear':
raise NotImplementedError("Got 5D input, but linear mode needs 3D input")
elif input.dim() == 5 and mode == 'bilinear':
raise NotImplementedError("Got 5D input, but bilinear mode needs 4D input")
elif input.dim() == 5 and mode == 'trilinear':
return torch._C._nn.upsample_trilinear3d(input, _output_size(3), align_corners)
else:
raise NotImplementedError("Input Error: Only 3D, 4D and 5D input Tensors supported"
" (got {}D) for the modes: nearest | linear | bilinear | trilinear"
" (got {})".format(input.dim(), mode))
Now the error is Could you help me?

[Naoto Inoue]

Thank you for reporting! I'm sorry to update the README.md. The minimum requirement for PyTorch is 0.4.1. Could you install it and try again?

[123liluky]

After installing torch0.4.1, new error is

And I can't find out how to solve it.

[Naoto Inoue]

Could you pull the latest master and try again? I haven't faced such problems currently.

[123liluky]

Thanks for your answering.

1. I found above error was caused by setting --image_size=256 in train.py. Now I am wondering the meaning of the third and forth line of output images in snapshots/default/images.
2. Besides I use 2242243 training images and 2242241 masks. Is it OK? I found generate_data.py generates 3 chanel masks in which pixel value is in [0,255]. But pixels in my masks is 0 or 255, without other values in (0,255).
3. Is the code suitable for inpainting images covered by big maks like below?
   
   I found another code is only suitable for inpainting very small masks like below
   

[Naoto Inoue]

1. I've added the description here.
2. So you mean there are non-black or non-white pixels in the masks and it might harm the performance, right?
3. Please refer to the original paper by the original authors. Personally I think its possible

[123liluky]

After training, I tested the model and got results like below
input image of network:

mask:

inpainting results:

The result is blurry. Do you have any suggestions to generate more plausible results.
loss_tv curve in tensorboard is like below:

[Naoto Inoue]

One general advice is that the hyper-parameters that the author reported are for datasets of natural images like Places2, they might be not suitable for your setting.
I'm sorry if there still remains a bug.

[Naoto Inoue]

In addition, the style/content loss is calculated based on pre-trained VGG. I'm not sure using these losses works for your grayscale images.