dolfly / ws-scrcpy

Web client prototype for scrcpy.

ws scrcpy

Web client for Genymobile/scrcpy and more.

Requirements

Browser must support the following technologies:

• WebSockets
• Media Source Extensions and h264 decoding;
• WebWorkers
• WebAssembly

Server:

• Node.js v10+
• node-gyp (installation)
• adb executable must be available in the PATH environment variable

Device:

• Android 5.0+ (API 21+)
• Enabled adb debugging
• On some devices, you also need to enable an additional option to control it using keyboard and mouse.

Build and Start

Make sure you have installed node.js, node-gyp and build tools

git clone https://github.com/NetrisTV/ws-scrcpy.git
cd ws-scrcpy

## For stable version find latest tag and switch to it:
# git tag -l
# git checkout vX.Y.Z

npm install
npm start

Supported features

Android

Screen casting

The modified version of Genymobile/scrcpy used to stream H264-video, which then decoded by one of included decoders:

Mse Player

Based on xevokk/h264-converter. HTML5 Video.
Requires Media Source API and video/mp4; codecs="avc1.42E01E" support. Creates mp4 containers from NALU, received from a device, then feeds them to MediaSource. In theory, it can use hardware acceleration.

Broadway Player

Based on mbebenita/Broadway and 131/h264-live-player.
Software video-decoder compiled into wasm-module. Requires WebAssembly and preferably WebGL support.

TinyH264 Player

Based on udevbe/tinyh264.
Software video-decoder compiled into wasm-module. A slightly updated version of mbebenita/Broadway. Requires WebAssembly, WebWorkers, WebGL support.

WebCodecs Player

Decoding is done by browser built-in (software/hardware) media decoder. Requires WebCodecs support. At the moment, available only in Chromium and derivatives.

Remote control

• Touch events (including multi-touch)
• Multi-touch emulation: CTRL to start with center at the center of the screen, SHIFT + CTRL to start with center at the current point
• Mouse wheel and touchpad vertical/horizontal scrolling
• Capturing keyboard events
• Injecting text (ASCII only)
• Copy to/from device clipboard
• Device "rotation"

File push

Drag & drop an APK file to push it to the /data/local/tmp directory. You can install it manually from the included xtermjs/xterm.js terminal emulator (see below).

Remote shell

Control your device from adb shell in your browser.

Debug WebPages/WebView

/docs/Devtools.md

File listing

• List files
• Upload files by drag & drop
• Download files

iOS

Experimental Feature: is not built by default (see custom build)

Screen Casting

Requires ws-qvh available in PATH.

MJPEG Server

Enable USE_WDA_MJPEG_SERVER in the build configuration file (see custom build).

Alternative way to stream screen content. It does not require additional software as ws-qvh, but may require more resources as each frame encoded as jpeg image.

Remote control

To control device we use appium/WebDriverAgent. Functionality limited to:

• Simple touch
• Scroll
• Home button click

Make sure you did properly setup WebDriverAgent. WebDriverAgent project is located under node_modules/appium-webdriveragent/.

You might want to enable AssistiveTouch on your device: Settings/General/Accessibility.

Custom Build

You can customize project before build by overriding the default configuration in build.config.override.json:

• INCLUDE_APPL - include code for iOS device tracking and control
• INCLUDE_GOOG - include code for Android device tracking and control
• INCLUDE_ADB_SHELL - remote shell for android devices (xtermjs/xterm.js, Tyriar/node-pty)
• INCLUDE_DEV_TOOLS - dev tools for web pages and web views on android devices
• INCLUDE_FILE_LISTING - minimalistic file management
• USE_BROADWAY - include Broadway Player
• USE_H264_CONVERTER - include Mse Player
• USE_TINY_H264 - include TinyH264 Player
• USE_WEBCODECS - include WebCodecs Player
• USE_WDA_MJPEG_SERVER - configure WebDriverAgent to start MJPEG server
• USE_QVH_SERVER - include support for ws-qvh
• SCRCPY_LISTENS_ON_ALL_INTERFACES - WebSocket server in scrcpy-server.jar will listen for connections on all available interfaces. When true, it allows connecting to device directly from a browser. Otherwise, the connection must be established over adb.

Run configuration

You can specify a path to a configuration file in WS_SCRCPY_CONFIG environment variable.

Configuration file format: Configuration.d.ts.

Configuration file example: config.example.yaml.

Known issues

• The server on the Android Emulator listens on the internal interface and not available from the outside. Select proxy over adb from the interfaces list.
• TinyH264Player may fail to start, try to reload the page.
• MsePlayer reports too many dropped frames in quality statistics: needs further investigation.
• On Safari file upload does not show progress (it works in one piece).

Security warning

Be advised and keep in mind:

• There is no encryption between browser and node.js server (you can configure HTTPS).
• There is no encryption between browser and WebSocket server on android device.
• There is no authorization on any level.
• The modified version of scrcpy with integrated WebSocket server is listening for connections on all network interfaces (see custom build).
• The modified version of scrcpy will keep running after the last client disconnected.

Related projects

• Genymobile/scrcpy
• xevokk/h264-converter
• 131/h264-live-player
• mbebenita/Broadway
• DeviceFarmer/adbkit
• xtermjs/xterm.js
• udevbe/tinyh264
• danielpaulus/quicktime_video_hack

scrcpy websocket fork

Currently, support of WebSocket protocol added to v1.19 of scrcpy

• Prebuilt package
• Source code