[bug] Cannot screen share a window in Flutter web app

[TPF]

Describe the bug

When running in Flutter web, attempt to share a window fails with error Could not start video source

Sharing a tab (in Chrome) and sharing the whole screen work as expected.

App Code

participant.setScreenShareEnabled(
  true,
  captureScreenAudio: true,
);

Call Stack

local.dart:478 
setSourceEnabled()

local.dart:530
final tracks = await LocalVideoTrack.createScreenShareTracksWithAudio(captureOptions);

video.dart:211
final stream = await LocalTrack.createStream(options);

local.dart:173
stream = await rtc.navigator.mediaDevices.getDisplayMedia(constraints);

jsutil function that errors

mediadevices_impl.dart:47
getDisplayMedia()

 @ line 55
final jsStream = await jsutil.promiseToFuture<html.MediaStream>(jsutil.callMethod(mediaDevices, 'getDisplayMedia', [arg]));

Exception:

DomException (NotReadableError: Could not start video source)

Flutter console log:

SEVERE: TLKController: _activateScreenShareWeb(): Web could not publish video: Unable to getDisplayMedia: NotReadableError: Could not start video source

To Reproduce

1. Run app on web (I'm using Chrome)
2. Commence screen share with call to Participant.setScreenShareEnabled()
3. Select 'window' tab and select a window then hit 'share'
4. Observe error in console.

Expected behavior

Expect the screen share to commence as it does for a shared tab or whole-screen share.

Platform information

Flutter Version

[✓] Flutter (Channel beta, 3.18.0-0.2.pre, on macOS 14.0 23A344 darwin-arm64, locale en-NL) • Flutter version 3.18.0-0.2.pre on channel beta at /Users/tpf-pricemoov/fvm/versions/3.18.0-0.2.pre • Upstream repository https://github.com/flutter/flutter.git • Framework revision fed06b31d9 (7 weeks ago), 2023-12-13 14:53:33 -0800 • Engine revision 39c6dc9bdd • Dart version 3.3.0 (build 3.3.0-174.3.beta) • DevTools version 2.30.0

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
• Android SDK at /Users/tpf-pricemoov/Library/Android/sdk
• Platform android-34, build-tools 34.0.0
• ANDROID_HOME = /Users/tpf-pricemoov/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
• All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 15C500b
• CocoaPods version 1.14.3

[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] VS Code (version 1.86.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.82.0

[✓] Connected device (2 available)
• macOS (desktop) • macos • darwin-arm64 • macOS 14.0 23A344 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 121.0.6167.139

[✓] Network resources
• All expected network resources are available.

• No issues found!

• Plugin version: 1.5.6
• Flutter target OS: Web/Chrome (simulator & prod)
• Flutter target OS version: N/A

[CloudWebRTC]

hey @T-P-F , I did several rounds of testing. I did reproduce the issue, but only after
Recurred when flutter run -d chrome,

but when I use flutter run -d web-server --web-port 8087 and open http://localhost:8087/, or use flutter build web && cd build/web && serve to open http://localhost:3000. At this time, All windows are shared correctly, and there will be no errors in the Chrome console.

So I guess that flutter run -d chrome opens an independent instance of chrome in sandbox mode, and it does not have permission to access windows other than itself.

[TPF]

Thanks for the effort on this @cloudwebrtc 🙏

I'm seeing this issue in production between two instances of Chrome. Each launched normally (not launched through an IDE).

[TPF]

This issue is resolved with the following:

1. Update to livekit_client v2.0.1+
2. On MacOS run tccutil reset ScreenCapture com.google.Chrome then regrant screen recording permission to Chrome. (reference)
3. Don't attempt screen share of a window or entire screen from inside Chrome if it was launched by VSCode.
4. Also made some modifications to my implementation that might have been related in this area when room participants are updated:

    for (var participant in room.remoteParticipants.values) {
      for (var t in participant.videoTrackPublications) {
        if (t.isScreenShare) {
          if (t.track != null && t.track!.isActive) {
            screenTracks.add(TLKParticipantTrack(
              participant: participant,
              videoTrack: t.track,
              isScreenShare: true,
            ));
          }
    }

Similar for the local participant.

HTH