SBTUITestTunnel not compatible with SwiftUI app set up
oliverfoggin opened this issue · comments
Following this issue #156
I created a brand new project in Xcode 14 as a SwiftUI app. I have a repo here... https://github.com/oliverfoggin/SBTTest-Broken-Tests
I added an AppDelegate (to replicate our actual app but also to takeOff SBTUITestTunnel) and added a test to download a small JSON payload and check for a label.
I am using a local mock server so you may need to change the URL to match your own set up.
When I run the test I can see the network hit my local server instead of being stubbed.
Also, if I change the AppDelegate to run takeOff on the first line (as documented) it crashes as the function seems to jump past the didFinishLaunching... and jumps into the view body which uses an implicitly unwrapped optional in the AppDelegate which hasn't had chance to be set up yet.
OK, I've updated to using a SceneDelegate and AppDelegate and changed the start up of the app to a more old school version and this still doesn't work.
It looks like the SBTUITestTunnel wants to completely take over the start up of the app. So if there's any property of the AppDelegate that needs to be passed into the ap on launch it just can't happen.
Hmm... I've made the property a let on the AppDelegate and it still doesn't work. SBTUITestTunnel can only stub the network call AFTER the app starts.
This is my app delegate did finish launching...
func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]? = nil
) -> Bool {
defer {
print("🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching defer")
}
print("🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching start")
#if DEBUG
SBTUITestTunnelServer.takeOff()
#endif
print("🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching mid")
viewStore.send(.didFinishLaunching)
print("🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching end")
return true
}
This is my test...
func testInvalidVersions() {
app.launchTunnel(withOptions: [SBTUITunneledApplicationLaunchOptionResetFilesystem]) { [weak self] in
let base_url = "http://0.0.0.0:9292"
let path = "/versions"
let complete_url = base_url + path
let method = "GET"
let status_code = 200
let requestMatch = SBTRequestMatch(url: complete_url, method: method)
let response = SBTStubResponse(fileNamed: "invalid_version.json", returnCode: status_code)
self?.app.stubRequests(matching: requestMatch, response: response)
}
Scenario("User is on an old app version and sees correct screen") {
Then(state(of: updateAppScreen.trait, is: .hittable))
}
}
When I run the tests and the look at the logs in the app it looks like this...
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching start
...
LOADS OF LOGS SHOWING THE APP STARTING AND HITTING THE NETWORK
...
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching mid
received action: AppFeature.Action.didFinishLaunching
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching end
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching defer
When I run the app normally and then look at the logs it looks like this...
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching start
[SBTUITestTunnel] required environment parameters missing, safely landing
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching mid
received action: AppFeature.Action.didFinishLaunching
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching end
🍎🍎🍎🍎🍎🍎🍎🍎🍎 Did finish launching defer
...
LOADS OF LOGS SHOWING THE APP STARTING AND HITTING THE NETWORK
...
From this you can see that the SBTUITestTunnel start up seems to be changing the way the app starts out.
I'm not sure if I'm doing something wrong or if the SBTUITestTunnel start up is changing things during start up 😅
Any help would be appreciated. Thanks
The test logs look like this...
Running tests...
Test Suite 'Selected tests' started at 2023-01-27 11:05:24.189
Test Suite 'xxxxxxx-AppUITests.xctest' started at 2023-01-27 11:05:24.189
Test Suite 'InvalidVersionTests' started at 2023-01-27 11:05:24.189
Test Case '-[xxxxxxx-AppUITests.InvalidVersionTests testInvalidVersions]' started.
t = 0.00s Start Test at 2023-01-27 11:05:24.190
t = 0.03s Set Up
t = 0.03s Open xxxxxxx
t = 0.05s Launch xxxxxxx
t = 0.05s Terminate xxxxxxx
t = 2.77s Setting up automation session
t = 3.34s Wait for xxxxxxx to idle
[SBTUITestTunnel] Did connect after, 4.479547s
[SBTUITestTunnel] Did perform startupBlock
[SBTUITestTunnel] Tunnel ready after 4.483921s
t = 4.51s User is on an old app version and sees correct screen
t = 4.51s Find the "Update app" Button
t = 5.54s Find the "Update app" Button (retry 1)
t = 6.59s Find the "Update app" Button (retry 2)
t = 7.63s Find the "Update app" Button
t = 8.67s Find the "Update app" Button (retry 1)
t = 9.70s Find the "Update app" Button (retry 2)
t = 10.75s Find the "Update app" Button
t = 11.79s Find the "Update app" Button (retry 1)
t = 12.83s Find the "Update app" Button (retry 2)
t = 13.87s Find the "Update app" Button
t = 14.92s Find the "Update app" Button (retry 1)
t = 15.96s Find the "Update app" Button (retry 2)
The app seems to start before the stubs happen.
Indeed there seems to be issues running the library with SwiftUI. I pushed some changes to https://github.com/Subito-it/SBTUITestTunnel/tree/debug/swiftui which I encourage you to checkout. From the quick preliminary checks it seems that it is now working as expected on the project you provided, but I still need to carefully review the changes.
Great! Thanks 😄
Can we close the issue?