abrarrobotics / TFClassify-Unity

An example of using Tensorflow with Unity for image classification and object detection

What

This is an example of using model trained with TensorFlow in Unity application for image classification and object detection. It's a quick port of TF Classify and TF Detect examples from TensorFlow repo, using TensorFlowSharp for gluing it all together.

Classify results:

Detect results:

Note that performance is worse than in TensorFlow Android example and at this moment I'm not quite sure how to improve that. Hopefully this will be enough to get you started.

How

You'll need Unity 2017.1 or above and Unity TensorFlow Plugin.

• Open the project in Unity.
• Install TensorFlow plugin.
• Open Classify or Detect scene in Assets folder.
• In Edit -> Player Settings -> Other settings add ENABLE_TENSORFLOW to the Scripting Define Symbols for the target platform.
• In Other settings also set Scripting runtime version to Experimental (.NET 4.6 Equivalent).
• Build and run.

More info can be found here.

If you want to use your own model, simply rename your file extension from .pb to .bytes, put it in Resources and set Model File and Labels File to your model and labels in main camera object of the scene you chose.

Notes

I'm neither Unity nor TensorFlow expert, so if you found any problems with this example feel free to open an issue.