ARKit Streamer

A remote debugging tool for AR Founndation with ARKit4 featrues. This is temporary solution until the Unity team has completed the AR remote functionality - See Unity forum for more information.

Human segmantation

Sample with VFX Graph

Environments

Tested on Unity 2020.3.1f1
ARFoundation 4.2.1
iPhone X or more

Supporting ARKit features

Basic camera position tracking
Send camera image via NDI
Human Segmentation / Depth
Face
Plane tracking
3D body tracking

How to Install

Install NDI SDK

Depends on NDI (Network Device Interface), download the NDI SDK for iOS from https://ndi.tv/sdk/

Install dependencies

Open the file Packages/manifest.json and add following lines into scopedRegistries and dependencies section.

{
  "scopedRegistries": [
    {
      "name": "Unity NuGet",
      "url": "https://unitynuget-registry.azurewebsites.net",
      "scopes": [ "org.nuget" ]
    },
    {
      "name": "npm",
      "url": "https://registry.npmjs.com",
      "scopes": [
        "jp.keijiro",
        "com.koki-ibukuro"
      ]
    }
  ],
  "dependencies": {
    "com.koki-ibukuro.arkitstream": "0.5.1",
    ...// other dependencies
  }
}

How to Use

Set up Sender iPhone

Download this reposidoty and install on iPhone. Then run the app on iPhone.

Or you can simply add ARKitSender to your custom ARKit scene.

Open the project settings and enable "ARKit Stream" as an XR plug-in for Unity Editor.

Recomend using USB connected network instead of Wi-Fi to reduce network delay.

Simurate on Editor

Add ARKitReceiver to the scene which you want to simuirate on Editor. Make sure the iPhone's IP Address and port are correct.

See the Assets/Sample for more infomation.

Setting for LWRP / URP

If you use LWRP / URP, you need also add the ARBackgroundRemoteRendererFeature to the list of render features. See AR Foundation Document for more information.

Libraries

KlakNDI

Rivadia / ARKitStreamer