`eval_and_inspect_test` running on web device failing on bots
elliette opened this issue · comments
On web, it passes but then hangs indefinitely and requires ctrl+c to quit the test:
On the CI bots, it fails: https://github.com/flutter/devtools/actions/runs/7730688845/job/21076721585?pr=7144
Logs when running locally
dart run integration_test/run_tests.dart --target=integration_test/test/live_connection/eval_and_inspect_test.dart --test-app-device=chrome
ChromeDriver - Starting ChromeDriver 120.0.6099.109 (3419140ab665596f21b385ce136419fde0924272-refs/branch-heads/6099@{#1483}) on port 4444
ChromeDriver - Only local connections are allowed.
ChromeDriver - Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver - ChromeDriver was started successfully.
FlutterDriveProcess - Resolving dependencies...
FlutterDriveProcess - _fe_analyzer_shared 61.0.0 (67.0.0 available)
FlutterDriveProcess - analyzer 5.13.0 (6.4.1 available)
FlutterDriveProcess - archive 3.4.9 (3.4.10 available)
FlutterDriveProcess - build_resolvers 2.4.1 (2.4.2 available)
FlutterDriveProcess - build_runner 2.4.7 (2.4.8 available)
FlutterDriveProcess - build_runner_core 7.2.11 (7.3.0 available)
FlutterDriveProcess - built_value 8.8.0 (8.9.0 available)
FlutterDriveProcess - code_builder 4.8.0 (4.10.0 available)
FlutterDriveProcess - coverage 1.7.1 (1.7.2 available)
FlutterDriveProcess - dart_style 2.3.2 (2.3.4 available)
FlutterDriveProcess - dds_service_extensions 1.6.2 (1.7.0 available)
FlutterDriveProcess - ! devtools_app_shared 0.0.9 from path ../devtools_app_shared (overridden)
FlutterDriveProcess - ! devtools_extensions 0.0.13 from path ../devtools_extensions (overridden)
FlutterDriveProcess - ! devtools_shared 6.0.4 from path ../devtools_shared (overridden)
FlutterDriveProcess - ! devtools_test 2.32.0-dev.8 from path ../devtools_test (overridden)
FlutterDriveProcess - ffi 2.1.0 (2.1.2 available)
FlutterDriveProcess - file_selector 1.0.1 (1.0.2 available)
FlutterDriveProcess - file_selector_android 0.5.0+4 (0.5.0+7 available)
FlutterDriveProcess - file_selector_ios 0.5.1+7 (0.5.1+8 available)
FlutterDriveProcess - file_selector_platform_interface 2.6.1 (2.6.2 available)
FlutterDriveProcess - flutter_lints 2.0.3 (3.0.1 available)
FlutterDriveProcess - flutter_markdown 0.6.18+2 (0.6.18+3 available)
FlutterDriveProcess - flutter_riverpod 2.0.0-dev.9 (2.4.9 available)
FlutterDriveProcess - http 1.1.2 (1.2.0 available)
FlutterDriveProcess - image 4.1.3 (4.1.4 available)
FlutterDriveProcess - intl 0.18.1 (0.19.0 available)
FlutterDriveProcess - js 0.6.7 (0.7.0 available)
FlutterDriveProcess - lints 2.1.1 (3.0.0 available)
FlutterDriveProcess - markdown 7.1.1 (7.2.1 available)
FlutterDriveProcess - mime 1.0.4 (1.0.5 available)
FlutterDriveProcess - mockito 5.4.3 (5.4.4 available)
FlutterDriveProcess - path_provider_platform_interface 2.1.1 (2.1.2 available)
FlutterDriveProcess - plugin_platform_interface 2.1.7 (2.1.8 available)
FlutterDriveProcess - pointer_interceptor 0.9.3+7 (0.10.1 available)
FlutterDriveProcess - pointycastle 3.7.3 (3.7.4 available)
FlutterDriveProcess - riverpod 2.0.0-dev.9 (2.4.9 available)
FlutterDriveProcess - shared_preferences_foundation 2.3.4 (2.3.5 available)
FlutterDriveProcess - shared_preferences_platform_interface 2.3.1 (2.3.2 available)
FlutterDriveProcess - source_gen 1.4.0 (1.5.0 available)
FlutterDriveProcess - state_notifier 0.7.2+1 (1.0.0 available)
FlutterDriveProcess - test 1.24.9 (1.25.2 available)
FlutterDriveProcess - test_api 0.6.1 (0.7.0 available)
FlutterDriveProcess - test_core 0.5.9 (0.6.0 available)
FlutterDriveProcess - url_launcher 6.2.1 (6.2.4 available)
FlutterDriveProcess - url_launcher_android 6.2.0 (6.2.2 available)
FlutterDriveProcess - url_launcher_ios 6.2.1 (6.2.4 available)
FlutterDriveProcess - url_launcher_linux 3.1.0 (3.1.1 available)
FlutterDriveProcess - url_launcher_platform_interface 2.2.0 (2.3.1 available)
FlutterDriveProcess - url_launcher_web 2.2.1 (2.2.3 available)
FlutterDriveProcess - url_launcher_windows 3.1.0 (3.1.1 available)
FlutterDriveProcess - vm_service 13.0.0 (14.0.0 available)
FlutterDriveProcess - vm_snapshot_analysis 0.7.2 (0.7.6 available)
FlutterDriveProcess - web_socket_channel 2.4.0 (2.4.3 available)
FlutterDriveProcess - win32 5.1.0 (5.2.0 available)
FlutterDriveProcess - xdg_directories 1.0.3 (1.0.4 available)
FlutterDriveProcess - xml 6.4.2 (6.5.0 available)
FlutterDriveProcess - Got dependencies!
FlutterDriveProcess - 52 packages have newer versions incompatible with dependency constraints.
FlutterDriveProcess - Try `flutter pub outdated` for more information.
FlutterDriveProcess - Launching integration_test/test/live_connection/eval_and_inspect_test.dart on Chrome in debug mode...
FlutterDriveProcess - Waiting for connection from debug service on Chrome... 36.9s
FlutterDriveProcess - This app is linked to the debug service: ws://127.0.0.1:61690/nYoYCJ_UnPI=/ws
FlutterDriveProcess - Debug service listening on ws://127.0.0.1:61690/nYoYCJ_UnPI=/ws
FlutterDriveProcess - 00:00 +0: (setUpAll)
FlutterDriveProcess - 00:00 +1: eval with scope in inspector window
ChromeDriver - [1706729325.630][WARNING]: This version of ChromeDriver has not been tested with Chrome version 121.
FlutterDriveProcess - devtools server not available (200)
FlutterDriveProcess - DevTools version 2.32.0-dev.8.
FlutterDriveProcess - No DTD uri provided from the server during initialization.
FlutterDriveProcess - TEST STATUS: verify that we can connect to an app
FlutterDriveProcess - Attempted to call extension 'ext.flutter.inspector.structuredErrors', but no service with that name exists
FlutterDriveProcess - getSourceReport: (-32602) Source report kind Coverage not supported
FlutterDriveProcess - TEST STATUS: switching to inspector screen (icon IconData(U+0F12B))
FlutterDriveProcess - TEST STATUS: testing basic evaluation
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#171b3),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS: testing variable assignment
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#8fb6d),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#3c3a9),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#e44cc),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS: testing eval on widget tree node
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with widget matching predicate descending from widget with key [<'Summary Tree'>]: [
FlutterDriveProcess - RichText(softWrap: wrapping at box width, overflow: ellipsis, maxLines: unlimited, text: "FloatingActionButton", dependencies: [Directionality, _LocalizationsScope-[GlobalKey#35bb9]], renderObject: RenderParagraph#5c590 relayoutBoundary=up2),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#c419f),
FlutterDriveProcess - ]
FlutterDriveProcess - 02:04 +2: (tearDownAll)
FlutterDriveProcess - 02:04 +3: All tests passed!
FlutterDriveProcess - result {"result":"true","failureDetails":[]}
FlutterDriveProcess - All tests passed.
FlutterDriveProcess - Application finished.
Logs when running on CI bot
FlutterDriveProcess - Launching integration_test/test/live_connection/eval_and_inspect_test.dart on Web Server in debug mode...
FlutterDriveProcess - Waiting for connection from debug service on Web Server... 78.8s
FlutterDriveProcess - integration_test/test/live_connection/eval_and_inspect_test.dart is being served at http://localhost:49649
FlutterDriveProcess - The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
FlutterDriveProcess - result {"result":"false","failureDetails":["{\"methodName\":\"eval and browse in inspector window\",\"details\":\"══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════\\nThe following TestFailure was thrown running a test:\\nExpected: exactly one matching candidate\\n Actual: _TextContainingWidgetFinder:<Found 0 widgets with text\\ncontaining Variable button is created : []>\\n Which: means none were found but one was expected\\n\\nWhen the exception was thrown, this was the stack:\\ndart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_\\npackages/matcher/src/expect/prints_matcher.dart.js 464:22 fail\\npackages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect\\npackages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$\\npackages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$\\neval_utils.dart.js 89:23 testEval\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 <fn>\\ndart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary\\ndart-sdk/lib/async/zone.dart 1308:19 runUnary\\ndart-sdk/lib/async/future_impl.dart 162:18 handleValue\\ndart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback\\ndart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners\\ndart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]\\ndart-sdk/lib/async/future_impl.dart 713:7 <fn>\\ndart-sdk/lib/async/zone.dart 1399:13 _rootRun\\ndart-sdk/lib/async/zone.dart 1301:19 run\\ndart-sdk/lib/async/zone.dart 1209:7 runGuarded\\ndart-sdk/lib/async/zone.dart 1249:23 callback\\ndart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop\\ndart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 <fn>\\n\\nThe test description was:\\n eval and browse in inspector window\\n═════════════════════════════════════════════════════════════════\\n\"}"]}
FlutterDriveProcess - Failure Details:
FlutterDriveProcess - Failure in method: eval and browse in inspector window
FlutterDriveProcess - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
FlutterDriveProcess - The following TestFailure was thrown running a test:
FlutterDriveProcess - Expected: exactly one matching candidate
FlutterDriveProcess - Actual: _TextContainingWidgetFinder:<Found 0 widgets with text
FlutterDriveProcess - containing Variable button is created : []>
FlutterDriveProcess - Which: means none were found but one was expected
FlutterDriveProcess -
FlutterDriveProcess - When the exception was thrown, this was the stack:
FlutterDriveProcess - dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_
FlutterDriveProcess - packages/matcher/src/expect/prints_matcher.dart.js 464:22 fail
FlutterDriveProcess - packages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect
FlutterDriveProcess - packages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$
FlutterDriveProcess - packages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$
FlutterDriveProcess - eval_utils.dart.js 89:23 testEval
FlutterDriveProcess - dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 <fn>
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1308:19 runUnary
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 162:18 handleValue
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 713:7 <fn>
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1399:13 _rootRun
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1301:19 run
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1209:7 runGuarded
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1249:23 callback
FlutterDriveProcess - dart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop
FlutterDriveProcess - dart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop
FlutterDriveProcess - dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 <fn>
FlutterDriveProcess -
FlutterDriveProcess - The test description was:
FlutterDriveProcess - eval and browse in inspector window
FlutterDriveProcess - ═════════════════════════════════════════════════════════════════
FlutterDriveProcess -
FlutterDriveProcess - end of failure 1
FlutterDriveProcess -
FlutterDriveProcess -
FlutterDriveProcess -
FlutterDriveProcess - Application finished.
Unhandled exception:
Exception: Test 'eval and browse in inspector window' failed: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: exactly one matching candidate
Actual: _TextContainingWidgetFinder:<Found 0 widgets with text
containing Variable button is created : []>
Which: means none were found but one was expected
When the exception was thrown, this was the stack:
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_
packages/matcher/src/expect/prints_matcher.dart.js 464:22 fail
packages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect
packages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$
packages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$
eval_utils.dart.js 89:23 testEval
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 <fn>
dart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary
dart-sdk/lib/async/zone.dart 1308:19 runUnary
dart-sdk/lib/async/future_impl.dart 162:18 handleValue
dart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7 <fn>
dart-sdk/lib/async/zone.dart 1399:13 _rootRun
dart-sdk/lib/async/zone.dart 1301:19 run
dart-sdk/lib/async/zone.dart 1209:7 runGuarded
dart-sdk/lib/async/zone.dart 1249:23 callback
dart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 <fn>
The test description was:
eval and browse in inspector window
═════════════════════════════════════════════════════════════════
.
result {"result":"false","failureDetails":["{\"methodName\":\"eval and browse in inspector window\",\"details\":\"══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════\\nThe following TestFailure was thrown running a test:\\nExpected: exactly one matching candidate\\n Actual: _TextContainingWidgetFinder:<Found 0 widgets with text\\ncontaining Variable button is created : []>\\n Which: means none were found but one was expected\\n\\nWhen the exception was thrown, this was the stack:\\ndart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_\\npackages/matcher/src/expect/prints_matcher.dart.js 464:22 fail\\npackages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect\\npackages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$\\npackages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$\\neval_utils.dart.js 89:23 testEval\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 <fn>\\ndart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary\\ndart-sdk/lib/async/zone.dart 1308:19 runUnary\\ndart-sdk/lib/async/future_impl.dart 162:18 handleValue\\ndart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback\\ndart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners\\ndart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]\\ndart-sdk/lib/async/future_impl.dart 713:7 <fn>\\ndart-sdk/lib/async/zone.dart 1399:13 _rootRun\\ndart-sdk/lib/async/zone.dart 1301:19 run\\ndart-sdk/lib/async/zone.dart 1209:7 runGuarded\\ndart-sdk/lib/async/zone.dart 1249:23 callback\\ndart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop\\ndart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 <fn>\\n\\nThe test description was:\\n eval and browse in inspector window\\n═════════════════════════════════════════════════════════════════\\n\"}"]}
Failure Details:
Failure in method: eval and browse in inspector window
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: exactly one matching candidate
Actual: _TextContainingWidgetFinder:<Found 0 widgets with text
containing Variable button is created : []>
Which: means none were found but one was expected
When the exception was thrown, this was the stack:
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_
packages/matcher/src/expect/prints_matcher.dart.js 464:22 fail
packages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect
packages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$
packages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$
eval_utils.dart.js 89:23 testEval
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 <fn>
dart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary
dart-sdk/lib/async/zone.dart 1308:19 runUnary
dart-sdk/lib/async/future_impl.dart 162:18 handleValue
dart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7 <fn>
dart-sdk/lib/async/zone.dart 1399:13 _rootRun
dart-sdk/lib/async/zone.dart 1301:19 run
dart-sdk/lib/async/zone.dart 1209:7 runGuarded
dart-sdk/lib/async/zone.dart 1249:23 callback
dart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 <fn>
The test description was:
eval and browse in inspector window
═════════════════════════════════════════════════════════════════
#0 IntegrationTestRunner.run.runTest (package:devtools_shared/src/test/integration_test_runner.dart:138:11)
<asynchronous suspension>
#1 IntegrationTestRunner.run (package:devtools_shared/src/test/integration_test_runner.dart:143:5)
<asynchronous suspension>
#2 runFlutterIntegrationTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/test_infra/run/run_test.dart:66:5)
<asynchronous suspension>
#3 _runTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:61:3)
<asynchronous suspension>
#4 runOneOrManyTests (package:devtools_shared/src/test/integration_test_runner.dart:330:9)
<asynchronous suspension>
#5 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:37:3)
<asynchronous suspension>
Error: Process completed with exit code 255.