SBTUITestTunnel: Failed getting IPC proxy on v8.2.0
zbencz3 opened this issue · comments
I updated my previous setup from 6.6.4 to 8.2.0 using Cocoapods (v1.10.1)
Installing GCDWebServer 3.5.4
Installing SBTUITestTunnelClient 8.2.0 (was 6.6.4)
Installing SBTUITestTunnelCommon 8.2.0 (was 6.6.4)
Installing SBTUITestTunnelServer 8.2.0 (was 6.6.4)
The UI tests on 6.6.4 worked with no problem. The 8.2.0 version seems to have the following issue:
[SBTUITestTunnel] Failed getting IPC proxy, Error Domain=DTXIPCErrorDomain Code=1 "Connection <DTXIPCConnection: 0x60000142e460> is invalid." UserInfo={NSLocalizedDescription=Connection <DTXIPCConnection: 0x60000142e460> is invalid.} (NSInternalInconsistencyException)
Any tips on where to look for the solution?
Thanks,
Zsolt
Setup:
Big Sur 11.3.1, Xcode 12.5.1 (12E507), iPhone 12 (iOS 14.5) but same happens on other simulators also.
Issue happens consistently.
Same thing happens on 8.1.0 and 8.0.0. Could not point to 7.0.0 to verify.
Takeoff:
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
#ifdef DEBUG
if (Utilities.isRunningUITests) {
NSArray<NSString *> *arguments = [NSProcessInfo processInfo].arguments;
if (![arguments containsObject:@"skipReset"]) {
[[[LogoutCoordinator alloc] init] __performLogoutActions];
}
if (![arguments containsObject:@"enableAnimations"]) {
[UIView setAnimationsEnabled:NO];
}
[SBTUITestTunnelServer takeOff];
}
#endif
...
}
Tests start:
app.launchTunnel(withOptions: ["reset"])
Test log:
Test Case '-[XXXUITests.YYYUITests testFlow]' started.
t = 0.00s Start Test at 2021-11-09 11:00:59.135
t = 0.05s Set Up
2021-11-09 11:00:59.191625+0200 UITests-Runner[83894:11297376] [SBTUITestTunnel] Resolving bonjour service com.subito.test.83894.65814125919098
t = 0.06s Open com.appXXX
t = 0.11s Launch com.appXXX
t = 0.11s Terminate com.appXXX:83819
t = 4.39s Setting up automation session
The following line
[[self sendSynchronousRequestWithPath:SBTUITunneledApplicationCommandStartupCommandsCompleted params:@{}] isEqualToString:@"YES"];
seems to trigger the ipcProxy handler:
self.ipcProxy = [self.ipcConnection synchronousRemoteObjectProxyWithErrorHandler:^(NSError * _Nonnull error) {
[self shutDownWithErrorMessage:[NSString stringWithFormat:@"[SBTUITestTunnel] Failed getting IPC proxy, %@", error.description] code:SBTUITestTunnelErrorLaunchFailed];
}];
Can you see #126. Maybe its the same issue, you could try putting in a breakpoint where I did to see.
The test log is unexpected because the "Resolving bonjour service" string is no longer used in the current version of the library. It would be interesting to understand that. Anyhow you can opt out of IPC as described here #106 (comment)
Thanks for the feedback @tcamin
The Resolving bonjour service was a mistake on my part, I posted that sample after I switched back to 6.6.4, instead of the 8.2.0. My apologies.
I did add the key/value SBTUITestTunnelDisableIPC YES to the UI test target's plist, but it seems now it times out waiting for the app startup. Interesting that the simulator is started and waiting.
Here an example with the correct startup sequence.
Test Suite 'Selected tests' started at 2021-11-09 18:02:57.086
Test Suite 'XXXUITests.xctest' started at 2021-11-09 18:02:57.086
Test Suite 'SignUpUITests' started at 2021-11-09 18:02:57.087
Test Case '-[XXXUITests.SignUpUITests testSignUpFlow]' started.
t = 0.00s Start Test at 2021-11-09 18:02:57.088
t = 0.05s Set Up
2021-11-09 18:02:57.139448+0200 XXXTests-Runner[19748:11768716] [SBTUITestTunnel] Resolving connection on port 8385
t = 0.05s Open com.XXX
t = 0.09s Launch com.XXX
t = 3.16s Setting up automation session
t = 11.71s Wait for com.XXX to idle
2021-11-09 18:03:27.610350+0200 XXXUITests-Runner[19748:11768716] [SBTUITestTunnel] Shutting down with error: '[SBTUITestTunnel] Waiting for startup block completion timed out'
Does this happen on a sample project as well? I tried on an empty project and everything works as expected. You can also try running the testStartupCommands test in the SBTUITestTunnel project.
@tcamin the sample project seems to work correctly.
The testStartupCommands and others also complete successfully. I am not sure what could make the difference 🤔
If I switch back to 6.6.4 it works, after the upgrade to 8.x it does not. Same Big Sur 11.3.1 (20E241), same simulator, Xcode version 12.5.1 (12E507), with the SBTUITestTunnelDisableIPC disabled or enabled.
Test Case '-[SBTUITestTunnel_Tests.MiscellaneousTests testCustomCommand]' started.
t = 0.00s Start Test at 2021-11-11 13:06:40.258
t = 0.07s Set Up
2021-11-11 13:06:40.331574+0200 SBTUITestTunnel_Tests-Runner[55727:13365573] [SBTUITestTunnel] Resolving connection on port 3781
t = 0.07s Open com.tomascamin.SBTUITestTunnel-Example
t = 0.12s Launch com.tomascamin.SBTUITestTunnel-Example
t = 2.35s Setting up automation session
t = 4.46s Wait for com.tomascamin.SBTUITestTunnel-Example to idle
2021-11-11 13:06:45.289732+0200 SBTUITestTunnel_Tests-Runner[55727:13367696] [] nw_protocol_get_quic_image_block_invoke dlopen libquic failed
2021-11-11 13:06:45.385349+0200 SBTUITestTunnel_Tests-Runner[55727:13367692] [SBTUITestTunnel] Did connect after, 5.054372s
2021-11-11 13:06:46.318671+0200 SBTUITestTunnel_Tests-Runner[55727:13365573] [SBTUITestTunnel] Tunnel ready after 5.987693s
In my case
Test Case '-[XXXUITests.AddNewUITests testFlow]' started.
t = 0.00s Start Test at 2021-11-11 13:09:53.200
t = 0.04s Set Up
2021-11-11 13:09:53.248144+0200 XXXUITests-Runner[56675:13383113] [SBTUITestTunnel] Resolving connection on port 41669
t = 0.05s Open com.XXX
t = 0.09s Launch com.XXX
t = 3.45s Setting up automation session
t = 12.61s Wait for com.XXX to idle
2021-11-11 13:10:23.579165+0200UITests-Runner[56675:13383113] [SBTUITestTunnel] Shutting down with error: '[SBTUITestTunnel] Waiting for startup block completion timed out'
2021-11-11 13:10:23.579620+0200 UITests-Runner[56675:13383113] *** Assertion failure in -[SBTUITunneledApplication testTunnelClient:didShutdownWithError:], SBTUITunneledApplication.m:65
UITests.swift:96: error: -[UITests.UITests testFlow] : [SBTUITestTunnel] Waiting for startup block completion timed out (NSInternalInconsistencyException)
t = 30.85s Tear Down
Increasing the connectionTimeout doesn't help.
Same failure happens on the CI.
Unchecking the Debug executable doesn't seem to help.
I noticed others had this type of problem in the past on 6.6.4.
If you manage to spot the difference please let me know
@tcamin it was an oversight on my part. I relied on a process named SBTUITunneledApplicationLaunchEnvironmentBonjourNameKey to determine if I am running UI tests and only then start the SBTUITestTunnelServer, to avoid starting it for unit tests, but that seems not to be available anymore in the latest update when querying ProcessInfo.processInfo. I am using the SBTUITunneledApplicationLaunchEnvironmentPortKey now.
Thanks for you time and patience.