About ToF AR

ToF AR, Time of Flight Augmented Reality, is a toolkit library intended to aid in Unity application development for iOS and Android devices. It consists of a group of functionalities based on depth information from ToF / Lidar (light detection and ranging) sensor and so on.

As well as ToF AR, Unity and a compatible device with a ToF sensor are required to build and execute this sample application.

Please see the ToF AR Site on Sony Developer World for ToF AR downloads and development guides, sample applications, and a list of compatible devices.

Overview of ToF AR Samples AR

ToF AR Samples AR is a sample application that uses the functions of ToF AR (see below).

When you start the application, the sample list is displayed on the screen. Select a sample from the list and tap it to start it.

Tap the screen with four fingers to return to the top screen.

List of scenes

SimpleARFoundation

A sample that uses both TofAR and UnityARFoundation features.

This sample can be used to check the operation of the Hand, Mesh, and Modeling features.

Puppet

This sample displays hand puppets at the position of each hand and forearm.

Hand Decoration

Maps a pattern onto the back of your hand.

When you show the back of your hand to the camera a pattern will be mapped onto it.

Rock Paper Scissors

A Rock Paper Scissors game using ToF AR gesture recognition.

The game starts by recognizing a thumbs up, and an in-app voice will guide you through the game.

Juggling

A Juggling game using face recognition and hand recognition.

To start the game, follow the on-screen instructions to place your hands in the appropriate juggling position.

A ball will drop when the game starts.

Catch the ball and throw it up in the air towards your other hand by moving the hand holding the ball up.

When you have caught the balls 5 times, a new ball will be added.

BGChange

Recognizes and changes the sky.

The background changes when you do a thumbs up.

TextureRoom

Creates a mesh of the environment with 3D Capturing, and maps text or photos onto the mesh.

Three modes are available. Tap the lower right hand icon and select the mode from "Mode DropDown".

TextureAnimation mode: Maps text onto the mesh.
TextInput mode: Enter the text to map from the input field at the bottom left.
Stamp mode: Touch the "Add Stamp Button" and select photos from the camera roll. The photos will be mapped onto the mesh.

BallPool

Generate AR balls in a recognized space.

The sample starts with space recognition to create a 3D mesh.

When the Ball Toggle set to On, balls will appear from above.

When objects like hands, feet, or a person, enters the scene it will move the balls.

StepOn

Grass and flowers appear from the point of contact.

The sample starts with plane recognition of the space.

When hands or feet touch a wall or floor, animations of grass and flowers appearing is shown at the point of contact.

Component

The table below shows the relationships between the scenes in the sample application and the ToF AR components used by each scene. The scene names are arranged vertically and the component names are arranged horizontally. A check mark indicates that the component is used.

	ToF	Color	Mesh	Coordinate	Hand	Segmentation	Face	Modeling
SimpleARFoundation	✓	✓	✓	✓	✓
Puppet	✓	✓			✓
Hand Decoration	✓	✓			✓
Rock Paper Scissors	✓	✓			✓
Juggling	✓	✓			✓		✓
BGChange	✓	✓			✓	✓
TextureRoom	✓	✓				✓		✓
BallPool	✓	✓		✓		✓		✓
StepOn	✓	✓		✓		✓

Assets

ToF AR Samples AR provides the following assets.

TofArSamplesBasic

Sample scene scripts and resources are stored for each component.

TofArSettings

Prefabs and scripts are stored as the Configuration change UI used by each component.

File	Description
Settings.Prefab	Configuration change UI
XXXController.Prefab	Manage configuration changes for each component

Development environment

Build library

ToF AR is required in order to build. Please download the Toolkit from the ToF AR Site on Sony Developer World and import it. Please see Setting up AR Foundation in the ToF AR user manual for more information on how to set up AR Foundation.
If the project is opened before importing, a confirmation message for entering safe mode will appear depending on the settings.
If safe mode is entered, please import after exiting safe mode from the safe mode menu etc.

Documents

ToF AR Development documents are also available on Developer World.

ToF AR user manual for overview and usage
ToF AR reference articles for articles about each component
ToF AR API references

Verification environment

Operation was verified in the following environment:

Unity Version : 2021.3.18f1
ToF AR Version : 1.3.0
AR Foundation : 4.2.7

Notes

Be aware that recognizable hand gestures may have different meaning in countries/areas.
Prior cultural checks are advisable.

Contributing

We cannot accept any Pull Requests (PR) at this time. However, you are always welcome to report bugs and request new features by creating issues.

We have released this program as a sample app with a goal of making ToF AR widely available. So please feel free to create issues for reporting bugs and requesting features, and we may update this program or add new features after getting feedback.

haosizheng / tof-ar-samples-ar

Contents