Using `imageWithSmartInvert` with `isRecording = true/false` - fails everytime

[Parth Parikh]

I am using the imageWithSmartInvert with isRecording = true/false and the test always fails

assertSnapshot(matching: view, as: .imageWithSmartInvert)

My original image is:

when i run with isRecording=true it generated the Black/Golden image.
when i run it with isRecording=false the snapshot does not match reference.

.
.
.
failed - Snapshot does not match reference.

"/Users/abc/Library/Developer/CoreSimulator/Devices/ACC40F6C-1421-4D1B-9008-20C0EB7B81EE/data/Containers/Bundle/Application/14408B86-E69D-4390-9E8D-0A050D98F518/SmallAppHost.app/PlugIns/SnapshotTests.xctest/SnapshotTestsBundle.bundle/Snapshots/Tests/testLoading.1.png"
@+
"/Users/abc/Library/Developer/CoreSimulator/Devices/ACC40F6C-1421-4D1B-9008-20C0EB7B81EE/data/Containers/Data/Application/E4FA0EDE-9A8C-4B4A-9E7B-4FDDBDF81EDB/tmp/Tests/testLoading.1.png"

Newly-taken snapshot does not match reference.
.
.
.

i tried opening the image in above locations - both are same -

can you please have a look?

[Parth Parikh]

attaching the images from xcresult:

zoom into the centre of the difference image (first image) and you will see the difference

[Parth Parikh]

the difference is

[Nick Entin]

Hi Parth! Can you provide some more details about what you're snapshotting? Is that a simple UIImageView in the center of the screen? If you have a branch adding a test case to AccessibilitySnapshot that reproduces the issue, that would greatly help me investigate what's going wrong.

[Parth Parikh]

@NickEntin - i and @abinodhthomas have added a support to add the precision - parikhparth23@ac84776
plz have a look and merge it.

[Nick Entin]

If you're seeing this on subsequent runs on the same machine, it sounds like there's likely an underlying issue going on here. While adding precision might work around it, I'd like to see if we can figure out the root problem first. Can you provide a code sample that reproduces the issue?

[Parth Parikh]

@NickEntin i wish i could share the code or even a snippet. the code is a proprietary code.
i tried creating a sample app and it works without any issue.
do you mind merging the precision code to your repo? - if you do it i will be able to use this framework. And i will look into our code base why the image is failing.

the precision value by default is to match 100% - so it wont affect the existing or future users.

And i just confirmed with our team the photos above is not an image- it is actually drawing the circles in code.

so this is not a bug.

do you mind adding the precision PR to your branch please?

[Nick Entin]

There is now tolerance parameters available when using FBSnapshotTestCase as the snapshotting engine. Support for SnapshotTesting is tracked by #63.

That said, I would highly recommend looking into why your views are not able to pass consecutive snapshots on the same machine and only use precision for inter-machine comparison failures. Perhaps you need to disable animations, or something along those lines? The framework is not designed to support snapshotting non-deterministic rendering views.