spidgets-3dof

Example app and hardware integrations for AR in a web browser with 3 degrees of freedom

System Requirements
Installation
Usage
How It Works
Student, Researcher, and OEM Info
Calibration and Performance
Manual Build and Deploy

System requirements

Compatibility

	Windows	MacOS	Linux	Android* (e.g. Samsung DeX)
Rokid Air, Max	✅	✅	✅	✅
XREAL Light, Air, Air 2, and Air 2 Pro	❌	✅	✅	✅
Viture, RayNeo Nxtwear, Air 2	❌	❌	❌	✅

*Experimental Android setup uses the device's integrated IMU instead of headset drivers, and therefore doesn't require installation. If your use case is mobile-only, see spidgets-mobile for a simpler repo and instructions for Termux.

Setting up new drivers

Installation

Linux

Clone this repo and open it in a terminal
Run chmod u+x ./bin/ar-server
Place the headset upright on a flat surface
Run sudo ./bin/ar-server and wait for calibration

Optional: Follow the udev step here for non-su

Windows

Clone this repo and open it in a powershell terminal
Place the headset upright on a flat surface
Run bin/ar-server and wait for calibration (first time setup)

MacOS

Skip to the manual build

Android

Android skip to the webserver step in the manual build.

Optional: Some third party websites (e.g. YouTube) need this extension in order to bypass restrictions caused by iframe headers. This may violate third party TOS or void warranties. Use at your own risk. On Android, extensions seem to only be available with Kiwi or Yandex browsers.

Soon: When stable, we can host an easier download/install process on a CDN + option to run in background on startup

Usage

Run bin/ar-server in a terminal

Browse to http://localhost:8000

Experiment with simple HUDs by editing webroot/index.html. Stay tuned for more interactive examples and docs.

Options:

Change port with the --port flag e.g. bin/ar-server --port 3000
Pre-record 10 seconds of motion data to allow testing without a headset using the --record flag. See webroot/compat for mock data example.
Calibrate for drift with the --cal flag. Refer to the calibration section.

How it works

... with headset IMU driver (Windows, Linux, MacOS)

Rust driver euler_60 reads the raw sensor data from the glasses and outputs euler angles (i.e. roll, pitch, and yaw) at 60Hz. Source
Server-side JS process ar-server manages the connection, corrects for yaw drift, and exposes the euler angles on a socket.io connection. Also exposes functions like re-centering and power save mode
Front-end in webroot/index.html listens to the websocket and uses spidgets-core to position the widgets and convert euler angles to matrix3d calculations to simulate 3D space

... on Android

Front-end in webroot/index.html uses spidgets-core to position the widgets and convert deviceorientation events in the browser to matrix3d calculations to simulate 3D space

Student, Researcher, and OEM Info

Connect new IMU devices by replacing bin/euler_60 driver with your own stdout program binary

Expected format:

...
   0.03188  -0.48346  -0.00605
   0.03174  -0.48465  -0.00596
   0.03166  -0.48591  -0.00588
   0.03196  -0.48720  -0.00600
...

i.e.

{whitespace}{pitch radians}{whitespace}{yaw radians}{whitespace}{roll radians}

Printed at or above target refresh rate

Then go to installation. For dev support, open an issue or contact: hello@3rl.io

Manual Build

Windows, Linux: see if the euler angles from your glasses are printing to the console by running the below command. If not, build headset-utils and copy the new euler_60 binary to the same location.

bin/euler_60

MacOS do the same with this instead:

bin/euler_60_apple_silicon

Android doesn't need euler_60.

Then get the webserver ready. Requires Bun or NodeJS

Start ar-server.js. You may need sudo for the second step:

bun install
bun ar-server.js

then browse to http://localhost:8000

Deployment

Create the main executable for end users:

bun build ./ar-server.js --compile --outfile bin/ar-server

Calibration

Calibration is a one-minute process that measures yaw drift rate and saves it for later. This process will run automatically during first time setup. It can also be run manually like so:

bin/ar-server --cal

The headset should be placed upright on a flat stable surface for the full 60 seconds. Console messages will indicate when the process is complete and the UI is ready.

At this time we don't store config profiles per-headset, so calibration will need to be re-run if multiple headsets are used

Optional: If the headset is usually used at a different angle than upright (e.g. in a recliner), it would be more accurate to wear it during calibration. Measurement starts 5 seconds after the command to allow time to adjust. The user can move freely during the test but should be in roughly the same orientation at T+5 and T+60.

Performance tips

With Rokid glasses, use a 120Hz display mode (either 1920x1080 or x1200 is okay). 60Hz creates input lag
iframes of third party websites can be very GPU-intensive especially if they have a lot of divs. See webroot/widgets for ideas for building performant widgets. HTML custom elements are the cleanest pattern.
If you have a discrete GPU, you may want to change your OS settings to use it for the browser. Most browsers will use the iGPU by default

3rl-io / spidgets-3dof