Padding difference between snapshots taken one after the other with no changes

[adgifs]

I am currently struggling with some snapshots and I do not understand the reasoning for it. This has happened to a lesser extent before but someone else from the team generated the snapshot with a M1 and it passed on the pipeline so that did the job at the time.

The problem is, right now, regardless if the snapshot is generated in a M2 (my machine) or M1, it does not work. In fact, right after taking a snapshot, even though there was no code change, the snapshot does not match. The only difference between both is that one image has a top padding compared to the other.

let traits: [String: UITraitCollection] = [
    "dark": .init(userInterfaceStyle: .dark), "light": .init(userInterfaceStyle: .light),
]
_ = try? traits.map { traits in
    try assertSnapshot(
        matching: view(),
        as: .image(
            drawHierarchyInKeyWindow: false,
            perceptualPrecision: 0.98,
            layout: .device(config: .iPhone13Pro),
            traits: traits.value),
        named: "\(traits.key)",
        timeout: 1.0,
        file: #file,
        testName: #function,
        line: #line
    )
}

Why is this happening when nothing is being changed between tests and the environment is the same?

[yevgenPaykey]

we have similar situation when run on new Xcode 15 beta

[Yonas Kolb]

Seeing this too. It randomly inserts an offset at the top, maybe 25% of the time. Same machine, same iOS version. Xcode 14.3 and iOS 16.4.
Tried all sorts of things to lock down any safe areas but to no avail. As it only happens some of the time it seems like it might be a timing issue. Waiting for another layout pass or another main thread hop doesn't seem to help either though.

[Frédéric Helfer]

I'm having the same issue with Xcode 14.3 and iOS 16.4. It randomly adds a padding to the navbar.

[pawisoon]

Is there any solution for it? Its still present in Xcode 15.0.1 and iOS 17.0.1

[pawisoon]

ping @stephencelis @mbrandonw

[Pascal]

For me there is a difference if I run all Tests of a XCTestCase or one after another.
If I run one Test at a time the Padding is correct and when all are run the whole XCTestCase the padding is wrong.

[pawisoon]

My colleagues and I are all using M1 / M2 machines. We are all having the same development environment meaning Xcode Version, OS version and Simulator model and its OS version.

Xcode 15.0.1
MacOS 14.1.1
iOS 17.0.1 and iPhone 15 Pro Max simulator

Colleague A - records snapshots on their machine (using erased and fresh simulator) and commits them to repo.
Colleague B - checks out the repo and runs tests, and they all fail because of a padding issue.

In CI - situation is the same, tests are failing because of padding issue. Each time, simulator device state is fresh(erased device).

We tested it on multiple computers and nothing seems to fix the issue.

Our temporary fix is done by lowering the precision by 14%(!)

[Zeynal7]

This seems to be fixed in the latest Xcode RC - 15.1

[Miguel Ferrando]

@Zeynal7 unfortunately it's not fixed for us :(

[Miguel Ferrando]

Our team finally fixed this issue. This has been a PITA and seems to be a bug on the macOS/iOS simulator. This may not work for you but this is what worked for us.

We realized that the issue wasn't a snapshot issue but actually, the simulator rendering all view controllers with some y offset:

For us it was just one colleague rendering the app differently in the simulator so we tried everything on his Mac: clone again the repository, re-install Xcode, runtimes, simulators... nothing worked until he reapplied the English language in Language & Region settings in macOS System Settings. So weird.

Hope this helps someone!