beresandras / buggy-resizing-critique

A criticism of a recent paper on buggy image downsampling methods in popular image processing and deep learning libraries.

A Criticism of the Paper On Buggy Resizing Libraries

Update 1: The paper was revised and renamed along with the main figure (January 2022). Now it discusses the antialias flag for Tensorflow, and also area/box interpolation. With that, I believe my criticisms have been addressed, and the paper now offers a thorough overview on this topic. Great!

Update 2: An antialias flag (False by default) was added to torch.nn.functional.interpolate() in Pytorch 1.11, March 2022.

This repository contains:

• a Jupyter notebook for reproducing the aliased image downsampling fenomenon, as demonstrated in the On Buggy Resizing Libraries paper, which argues that the image downsampling methods of the OpenCV, Tensorflow and PyTorch libraries are "buggy", with only PIL being correct.
• simple solutions for antialiasing in every framework, which solves the issue in all cases using the same functions, simply by setting parameters appropriately:
  • OpenCV: change the interpolation from bilinear to area (from cv2.INTER_LINEAR to cv2.INTER_AREA)
  • Tensorflow: set the antialias flag to True
  • PyTorch: change the interpolation mode from bilinear to area, or simply use torchvision.transforms.Resize() instead of torch.nn.functional.interpolate()

Try it out in a Colab Notebook:

My opinion:

• neither of the used image downsampling methods is "buggy", not applying antialiasing by default is an understandable design decision for both image and tensor operations.
• the main figure of the paper is misleading, and it only illustrates the issues of aliasing for image resizing.
• the aliasing issue with downsampling can be solved in all frameworks by simply setting a few parameters correctly. My criticism is that this is not mentioned in the paper.
• torchvision.transforms.Resize() is claimed to only be a "a wrapper around the PIL library" in a note in Section 3.2 of the paper. This is true for PIL image inputs, but is incorrect for torch.Tensors, which are resized using torchvision interpolation operations.
• the remaining parts of the paper provide valuable insights into the effects of interpolation methods, quantization and compression on the FID score of generative models.

There is another, very thorough investigation of the same issue. Highly recommend checking the blogpost out. They also implement an OpenCV-compatible Pillow-equivalent resizing that provides proper antialiasing for all interpolations.

Bilinear downsampling results with and without aliasing:

The main figure (Figure 1) of the paper: