Sometimes (not often) I get wrong first visual change
soulgalore opened this issue · comments
Hey Pat,
this happens rarely, maybe 1 or 2 times of 30 tests that for some videos the orange change is picked up as first visual change. It could be that I use the wrong input parameters, I'm not sure or ffmpeg.
I use a Docker container just to make it easy to run (with the absolute latest source of VisualMetrics).
I've uploaded the video here: https://www.dropbox.com/s/ktvc9ephtkm8lq3/18.mp4?dl=0
The video starts with orange, then go white and then loads the page. I run like this:
docker run -v "$(pwd)":/tmp wikimedia/visualmetrics python visualmetrics.py --video /tmp/18.mp4 --orange -k --dir /tmp/images --force --white
And the First Visual Change is 17 because it picks up this as the first change:
As I said this works almost all the time for me but sometimes it happens.
Best
Peter
FYI: I had this problem on Ubuntu running ImageMagick 6.7.7-10 and updating to 6.9.7-1 fixed the problem.
No actually that only solved the problems sometimes. I still get the same thing. It happens only when I record the screen on Chrome (not Firefox). I'll check more if I can find the problem.
We have a slightly modified version of VisualMetrics, I wanna check if I can send a PR so we can make it configurable to handle both cases.
We have changed https://github.com/WPO-Foundation/visualmetrics/blob/master/visualmetrics.py#L673 to just skip the cropping (our videos are the right size at the beginning) because the cropping made us miss the orange part in the example above. We cropped out the orange and then firstVisualRender was calculated wrong. We just use:
command = ('convert "{0}" "(" "{1}" -resize 200x200! ")" miff:- | '
I've seen we should do the same change on where we decide if an image is white to make it the same.
We also made https://github.com/WPO-Foundation/visualmetrics/blob/master/visualmetrics.py#L193 and https://github.com/WPO-Foundation/visualmetrics/blob/master/visualmetrics.py#L215 larger just when we had more frames, maybe we can make that configurable too.
We also made the limit https://github.com/WPO-Foundation/visualmetrics/blob/master/visualmetrics.py#L679 configurable but I think the first fix of not cropping the image fixes that.
Would you accept a PR for making cropping and frames configurable? Then we can use Visual Metrics straight of + hopefully also help people that don't need that cropping.
I'm a little confused by how removing the crop would have fixed the screenshot above. Comparing the entire frame against the orange frame would still show it as not being completely orange unless you change the crop to look at the bottom part of the frame only. The crop just pulls a center chunk out of the image (50% width, 33% height) to see if that section is all orange and should be discarded.
I'm happy to accept PR's so you don't have to maintain patches, I just want to be sure that the changes make sense (more frames being configurable makes total sense).
Hi Pat, returning to this after of years :) I still see this problem and I don't fully understand the logic in the code:
https://github.com/WPO-Foundation/visualmetrics/blob/242348880b1219482b3ebeba28e6a7dcfc27190f/visualmetrics.py#L754-L788
We are comparing images that are 200*200 so we compare 40000 pixels. And if less than 100 of them are different the image is a color frame. But the idea with the check was to find a couple of lines of orange (the problem with Chrome not fully vertical syncing). In that case maybe a couple of thousand pixels are orange. Shouldn't be that if less than the 40000 pixels are different, it is still a match (=partly orange)?
Best
Peter