Giters

bmegli / unity-network-hardware-video-decoder

Video and point cloud streaming over custom UDP protocol with hardware decoding (Unity example)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

point cloud streaming delay (measurement)

bmegli opened this issue · comments

commented

Experiment

experiment_description_small

Glass-to-glass delay benchmark of WiFi streaming with:

  • LattePanda Alpha
    • Ubuntu 18.04
    • USB WiFi EW-7612UAn V2 (N standard), 2.4 GHz WiFi
    • Realsense D435 (FW 5.12.1.0, development)
    • RNHVE commit 6000648
  • wireless Router Asus RT-AC66U
  • laptop Alienware 15 R3 (2017, i7-7820HK)
    • 5GHz WiFi
    • Ubuntu 18.04
    • UNHVD commit 0d7277f
      • player built with 2019.3.0f3 Unity
      • DepthUnits configured to 0.000015625f (~ 1 m range, ~ 1 mm precision after P010LE encoding)

Tools:

  • Samsung S9+ (for slow motion recording)
  • ffmpeg (for extracting separate frames)

Methodology

  1. Run unity-network-hardware-video-decoder
  2. Run realsense-network-hardware-video-encoder (848x480, 30 fps, 4000000 bitrate)
./realsense-nhve-hevc 192.168.0.100 9768 depth 848 480 30 5000 /dev/dri/renderD128 4000000 0.000015625
  1. Capture slow motion video of ping-pong ball bouncing off table seen directly (mobile phone camera) and on screen where we are streaming as point cloud (laptop) over WiFi
  2. Download video to PC
  3. Extract separate frames with ffmpeg
ffmpeg -i video.mp4 -qscale 0 scene%05d.jpg
  1. Calculate number of frames between the moment when bouncing off table seen directly and on screen as point cloud
  2. Calculate glass-to-glass delay assuming one frame time 1000/240=4,16 ms (Samsung S9+ 240 fps slow motion)
  3. Repeat experiment 5 times, calculate mean and standard deviation

Example:
Ball bouncing off table in frame 877 directly
scene00877
Ball bouncing off table in frame 901 on screen
scene00901
The delay is 901-877=24 frames, 24*1000/240=100 ms

Methodology verification

  1. Stopwatch recorded in slow motion
  2. Single frames extracted with ffmpeg
  3. Confirmed that 1 second takes roughly 240 frames

Other

Related to #8
Edits: added photos, fixed typos, confirmed router model

commented

Results

# Frames Delay [ms]
1 24 100
2 24 100
3 29 120,8
4 28 116,7
5 30 125

Result 112.5 +- 12 ms

commented

Notes: