[QUESTION] getDifferencePercent meaning

[dmiroshnyk]

Hi @romankh3 !
Could you please clarify how the differencePercent is calculated?
I compare these 2 images:

and

and got the result:

Then I trying to use something like:
return (imageComparisonResult.getDifferencePercent() <= diffPercentage)

I expected getDifferencePercent is calculated like total square of all difference rectangles vs image square, like:
Sum (S<i = 1..n> S) / S ~ 15% for my images.

But surprisingly I got 0.0! Could you please clarify where I'm wrong?
Thanks in advance!

[Roman Beekeeper]

Hello @dmiroshnyk,
Thanks for your question.
Could you please provide full java code to ensure that I understood you correctly?

Best regards,
Roman.

[dmiroshnyk]

`
BufferedImage expectedImage = ImageComparisonUtil.readImageFromResources(expectedImagePath);

  BufferedImage actualImage = ImageComparisonUtil.readImageFromResources(actualImagePath);

  File resultDestination = new File(resultFilePath);

  ImageComparison imageComparison = new ImageComparison(expectedImage, actualImage, resultDestination);

  imageComparison = setupComparison(imageComparison, borderThreshold, rgbThreshold, excludedAreaCoordinates);

  ImageComparisonResult imageComparisonResult = imageComparison.compareImages();

  ImageComparisonUtil.saveImage(resultDestination, imageComparisonResult.getResult());

  report.report("Comparison difference: " + imageComparisonResult.getDifferencePercent());`

[Roman Beekeeper]

Please, provide setupComparison method implementation.

[dmiroshnyk]

It provides basic properties setup
` private static ImageComparison setupComparison (ImageComparison comparison, int borderThreshold, float rgbThreshold, String[] excludedAreaCoordinates) {
comparison.setThreshold(borderThreshold);
comparison.setRectangleLineWidth(1);
comparison.setDifferenceRectangleFilling(false, 20.0);
comparison.setDrawExcludedRectangles(true);
comparison.setExcludedRectangleFilling(false, 20.0);
comparison.setMinimalRectangleSize(1);
comparison.setPixelToleranceLevel(rgbThreshold);

LinkedList<Rectangle> excludedAreas = new LinkedList<Rectangle>(){
};
if (! Nullable.isNullOrEmpty(excludedAreaCoordinates)) {
  for (int i = 0; i < excludedAreaCoordinates.length; i += 4) {
    excludedAreas.add(new Rectangle(Integer.parseInt(excludedAreaCoordinates[i]),
            Integer.parseInt(excludedAreaCoordinates[i + 1]),
            Integer.parseInt(excludedAreaCoordinates[i + 2]),
            Integer.parseInt(excludedAreaCoordinates[i + 3])));
  }
}
comparison.setExcludedAreas(excludedAreas);
return comparison;

}
`

[Roman Beekeeper]

Hi @dmiroshnyk, I found what you're looking for.

To understand how it works, you can see this logic

/**
     * Return the difference in percent between two buffered images.
     *
     * @param img1 the first image.
     * @param img2 the second image.
     * @return difference percent.
     */
    public static float getDifferencePercent(BufferedImage img1, BufferedImage img2) {
        int width = img1.getWidth();
        int height = img1.getHeight();

        long diff = 0;
        for (int y = 0; y < height; y++) {
            for (int x = 0; x < width; x++) {
                diff += pixelDiff(img1.getRGB(x, y), img2.getRGB(x, y));
            }
        }
        long maxDiff = 3L * 255 * width * height;

        return (float) (100.0 * diff / maxDiff);
    }

For now, this logic is using in case, when we have SIZE_MISMATCH, that's why for valid image sizes this is not counting. From my perspective, this looks like a bug.

Do you really need it?

This is was added and maintained by the community and I haven't paid enough attention on it.

[dmiroshnyk]

Hi @romankh3 ,

Thanks for digging into it!

In case it's easy to fix - yes, I will be very appreciated to have a new version with fix - this allows my tests to be more manageable. Could you please share the estimated time to fix if possible? This allows me to understand my plans.

I also have several questions regarding this:

1. So, the diff is calculated only when images has different size, right? In case images have same size, the diff will equal 0.0?
2. This diff logic works for 1-st image as base one. In this case getDifferencePercent(img1, img2) != getDifferencePercent(img2, img1). This doesn't look like a good idea. I recommend using smallest dimensions (in case we want to see no diff if img2 is a part of img1) or biggest ones (in case we want to see diff in this case), like:
   int width = min(img1.getWidth, img2.getWidth); int height = min(img1.getHeight, img2.getHeight);
3. Why diff between pixels is calculated like (pixel2RGB - pixel1RGB)? For example, we have pixel2 - white. Diff vs pixel1 depends on pixel1 RGB color. So, diff vs red pixel will be less than vs blue. I don't understand the reason of this. For example, if I convert my image with google and change all red colors to blue and ask to recalculate diff (let's imagine the bug is fixed) - the diff percentage return another value which looks having no sense because the result file will look identically. IMHO, the diff between single pixel should be normalized to 0 (no diff) or 1(diff) and formula should look like: (<# of all different pixels> / <# of all pixels>) * 100. What do you think about it?
4. What is the difference between getDifferencePersent and allowingPercentOfDifferentPixels? It looks like a similar approach but in my example above I see MISMATCH but 0.0 difference, so it is calculated in another way.

[Roman Beekeeper]

It's not so hard, but I don't have enough time for it. As you can see, I answered after week.

1. no, the diff would be not 0.0 in case, when we have ImageComparisonState.MISMATCH
2. I got your point, but your idea doesn't help me. The main feature is to show place, where two images different. If you want to draw on image2, you can just switch arguments (img1, img2) to (img2, img1)
3. third - let's discuss in a different issue.
4. As I said, getDiffrentPixels send 0.0 almost all the time and must be added for all cases. allowingPercentOfDiffentPixels allows us to skip pixels in comparing.

[dmiroshnyk]

Hope you'll have time for this one day :)
Alternatively, I can try to fix this myself and make a pull.

1. Hmm... Maybe we have a misunderstanding there. I asked about how does it work now. In my first example, I have ImageComparisonState.MISMATCH but 0.0 difference.
2. Why not? If you want to show diff - you have to take bigger dimensions as base ones. So you will definitely have the difference to display between files with any dimensions. The only thing there this may change the result output size, like: img1 = 100X200, img2 = 50X400. Resulting image size will be 100X400 which may lead to issues if output is expected with img1 size.
3. OK, will move it to another issue.
4. Maybe I missing smth. Could you please clarify how the MISMATCH is calculated? As I stated in my 1-st answer, now I see MISMATCH with 0.0 difference and this misleads me because it looks MISMATCH is calculated basing on another algorithm.

[Roman Beekeeper]

Hi @dmiroshnyk,
Always you can add your contributions.

1. as I mentioned before, this is the bug, that if MISMATCH we have getDifference = 0.0
2. mismatch is calculating when we have rectangles with differences. In some configurations, we can skip some rectangles(such as maximalRectangleCount, minimalRectangleSize).

To summarise, if you can - you can fix this bug with getDifference() for MISMATCH image comparison state. But before you will start - we need to create BUG-ISSUE for it to trsck in future.

[Roman Beekeeper]

added it to 4.3.0 version