morgangiraud / OpenMined

The OpenMined Unity Application

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

OpenMined Unity Application

Build Status License Help Contribute to Open Source

Table of Contents:


OpenMined Unity Application applies the PySyft library into a Unity Application. Please see the PySyft repository for more details on the intent of OpenMined and to familiarize yourself more with the basic concepts of the project.

Also check out the Docs for more information! We are working toward consolidating all the documentation to make it easier for new contributors.

Why Unity?

Unity has a few advantages that we are leveraging for this project

  1. It's portable.
  2. It's free
  3. It has versatile GPU access.
  4. It can be used for high end graphics gaming consoles (Playstation, XBox, etc)

More about this in the corresponding file at tutorials folder

Marketplace Mechanics


Open Mined Architecture


If you have an issue, refer to the following steps for a more detailed project setup. These steps were confirmed to work under a Windows/linux environment, though the steps should be applicable to alternative operating systems.

The steps come in two parts:
Part 1: Unity Setup
Part 2: Jupyter Setup

Unity Setup

  1. Download Unity from

  2. Open project in Unity [Open(Top Right of Home Screen) -> Directory/To/OpenMined -> /UnityProject]

  3. In the Project Pane (usually below), double click OpenMinedMain. If you can't find the file for some reason, check the trouble shooting section.

  4. Hit Play on the Unity Editor

Jupyter Setup

  1. Run jupyter notebook from your OpenMined directory
  2. Open Syft Tensor Example Notebook.ipynb
  3. Run the notebook

General Troubleshooting

  1. If OpenMinedMain is missing
  • On the right menu: Check Main Camera object has SyftServer.cs component attached to it
  • On the bottom dialog: Go to OpenMined/Network/Servers drag SyftServer.cs to Main Camera object
  • Add a Compute Shader to the Shader variable of SyftServer.cs script
  • Go to OpenMined/Syft/Math/Shaders drag FloatTensorShaders to SyftServer (Script) component recently attached to Main Camera
  1. If my applications do not seem to be communicating between eachother...

Check if the Server is running...

It should run on port 5555 and this can be checked by running the following command on CMD with administrator permissions.

netstat -a -b | findstr :5555  

If just the Server is running, the output should be:


If both Server and Jupyter Notebook are running and communicating, the output should be:


Another way:

lsof -i :5555

Result should be:

Unity   1709 user   38u  IPv4 0x59e297c6d0d734e31      0t0  TCP *:personal-agent (LISTEN)

  1. My application randomly stops working.

Jupyter Notebook only works if Unity has focus
By default, the "Run in background" options is disabled. So if the Unity Editor loses focus then the Jupyter Notebook won't work. Go to Edit -> Project Settings -> Player. The inspector pane will now change to show the player settings. Look for the option that says "Run In Background" and check it [1]

Links and How Tos

How Tos:

How to add a Function to Float Tensor

Video Links:

Architecture Proposal


[1] stop unity pausing when it loses focus


The OpenMined Unity Application

License:Apache License 2.0


Language:C# 62.0%Language:Jupyter Notebook 32.6%Language:Python 5.0%Language:PowerShell 0.3%Language:Shell 0.2%Language:ShaderLab 0.1%