pupil-labs / pupil-community

Community contributed plugins and projects using Pupil

pupil-community

This repository is a README only repository for community contributed projects, forks, plugins, scripts, and videos using Pupil - open source eye tracking platform.

Contributing

Want to add your project, fork, plugin, or script to this list? Fork this repo and make a Pull Request with a link and short description in the README file.

Want to add your demo video to this list? Make an issue with the Youtube link and the maintainers will add the video to this Youtube playlist.

Plugins

Install instructions: https://docs.pupil-labs.com/developer/core/plugin-api/#adding-a-plugin

Pupil Detector Plugins

Requires Pupil v2.6 or higher.

• artificial_2d_pupil_detector.py: Example pupil detector plugin that generates artificial pupil data
• custom_2d_pupil_detector.py: Example pupil detector plugin that extends the default 2D detector
• all_controls_pye3d.py: Plugin that exposes pye3d's internal settings via the UI
• all_controls_2d.py: Plugin that exposes all 2D's internal settings via the UI

Video Backends

• RealSense D400 Source v1.22 - A port of the old realsense2 plugin (for RealSense D400 series cameras) to work with the new video source framework in Pupil Capture v1.22.
  • Up to Pupil Capture v2.2 (here)
  • From Pupil Capture v2.3 (here)
• Pupil Video Backend - Streams Video from another computer through the network to your computer where pupil capture software is running.
• Ximea Camera Plugin - A plugin to preview and record frames from Ximea Cameras using XiAPI.

Calibration Choreographies and Gazers

• Dual-monocular 2D and 3D Gazers - 2D and 3D gaze mapping plugins that map all pupil data monocularly. Recommended if when investigating strabismus or similar conditions. Requires Pupil v2.2 or higher.
• Dual-display HMD calibration choreography and gazer - Dual-monocular 2d calibration, but with the possibility to calibrate different target coordinate systems for each eye.
• Nine-point calibration choreography
• Custom calibration confidence threshold

Minimal Examples

• Custom Data example - Pupil Capture plugin that demonstrates how to publish custom data via the Network API and have it saved during a recording
• Online Task Proxy - Example plugin that shows how to use the Task_Proxy class in Pupil Capture
• Unix Time Sync - Example plugin that shows how to synchronize the Pupil clock to Unix time
• Depth Frame Accessor - Example plugin that shows how to access the 16 bit depth data of Realsense frames in Capture
• TaskLib Example - Example plugin that demonstrates basic task_lib usage.

Pupil Player Visualizations

• Single gaze point visualization - Pupil Player plugin that only visualizes the gaze point that is closest in time to the current frame
• Render frame index into video image - Pupil Player plugin that renders the current frame index into the video frame
• Render date time into video image - Pupil Player plugin that renders the recording time of the current frame
• Video overlay with manual temporal-alignment option - Allows to synchronize unaligned videos

Others

• LabStreamingLayer integration
• Annotations without log messages - Pupil Capture plugin that does not display a log message when receiving an annotation. This is helpful if you need to receive many annotations.
• Gaze from recording with manual offset correction
• Gaze-controlled VLC Player - gvlc - Gaze-controlled VLC player using Pupil Capture.
• Pupil Labs usybus Controller - This plugin exports pupillabs eye tracking gaze to Ivybus.
• Pupil progs - Python helper scripts used with Pupil (post-hoc and real-time).
• @cpicanco's Pupil Player Plugins - A collection of visualization and analysis plugins for Pupil Player.
• @cpicanco's Pupil Capture Plugins - A collection of plugins for Pupil Capture.
• Pupil 2D Detection Preview - Plugin which shows a scrollable gallery of eye images with 2D pupil detection after a recording stops. Its purpose is to get a brief visual insight of how good the detection will be, when the online detector was disabled due to perfomance or stability reasons. The eye images are saved with a fixed time interval in a subdirectory of the recording. It also shows the usage of changing the detection properties and accessing the GPOOL in a plugin. The frame publisher has to be enabled during recording.
• pupil-t265 - Calibrate, record and export tracking data from the Intel RealSense T265 tracking camera.
• Export PI template data - Pupil Player plugin to export template questions and answers from Pupil Invisible recordings
• Cursor Control - Use surface tracking and surface-mapped gaze to move your computer's mouse cursor.

Projects & Forks

• PyPlr - Many-featured Python library for researching the pupillary light reflex with Pupil Core. Easily time-stamp any light stimulus with the World Camera.
• Pupil Labs Motion Capture Trigger - Use a low-cost Arduino microcontroller to synchronise motion capture systems (e.g. Vicon/Qualisys) with the Pupil Core Mobile eye tracker.
• Plugin to Perceive the Object in Focus - Real-time object-detection plugin that integrates eye-tracking (fixation data) to identify which object the user is currently fixated upon. An example of its potential is shown in controlling grasps of a robotic prosthetic hand. Part of the Therabotics Project
• PolyMouse - PolyMouse is a fusion of gaze tracking and head tracking to achieve hands-free pointing. It does this by using eye gaze to quickly determine an imprecise area of interest and then uses small head movements to refine the position of the cursor onto a small target.
• Asistiva - a set of tools for creating amazing web apps optimized to be used through different input methods (i.e. including eye-tracking, head-tracking, button-less mouse, adapted switches, etc).
• Focal - Focal is an experimental eye-tracking musical expression controller which allows hands-free control over audio effects and synthesis parameters during performance. A see-through head-mounted display projects virtual dials and switches into the visual field.
• Graph viz eye tracker - Creating a gaze-aided graph navigating application using Unity3D and Pupil-lab.
• Optical Dimension Spatial Thinking - ODST - Gaze controlled interface using gaze data and voice recognition.
• PupilCamC++ - Interface for capture of Pupil labs cameras with C++, primarily for windows
• Pupil-lib Trial/Event Extraction Library - A flexible tool for conducting high precision event based research analysis with Pupil Labs eye trackers. See here for how to run an experiment that produces data compatible for this library. A Matlab version is also available.
• Mobile Gaze Mapping - Automatic gaze mapping between world camera and a fixed reference stimulus. This tool provides automatic detection of a 2D target stimulus in a mobile eye-tracking recording, and translates recorded gaze data from the world camera to the fixed coordinate system of the stimulus itself.
• JupilLabs - A Processing(Java) binding for extracting data from pupil-labs with jeromq and msgpack-java
• Save data where users looked - A script that allows to save data where the user has looked using Pupil in a Unity Application
• Luxometer - Added functionality to read data from a AdafruitTSL2591 via serial (linux only; via)

Scripts

Real-time Network API clients

• pupil-remote-php - A simple PHP remote control client for the Pupil eye tracker software.
• Lab Matlab Control - Custom utilities for experimental control.
• Pupil Pascal Client - Free Pascal requester and subscriber clients for Pupil.
• Pupil MatLab Client - MatLab subscriber client for Pupil.
• Pupil Middleman - A middleman client written in Python. This client can be used to interface with Pupil Capture and other sytems and languages that do not natively support ZeroMQ.
• Network Topic Monitor - A very simple network topic monitor. Shows notification count for every topic and how long ago the last notification was received.
• OptiTrack and Pupil Labs Python Recorder - A simple python script that records data from Pupil Labs and OptiTrack.
• Surface OSC Bridge - Redirects surface data using the OSC protocol.
• IPC Backend Simulator - Simulates the Network API of a Pupil Capture instance, publishing data from a provided recording

Post-hoc Analyses

• Generate Pupil-Player-compatible recording from externally recorded video
• Offline Gaze Mapping Tools for Pupil Labs Glasses - Offline Gaze Mapping Tools for Pupil.
• Eye Tracking Research Data Analysis - Python pupillometry analysis scripts to analyize pupil_data.
• extract_diameter.py - Extract 2d and 3d (if available) pupil diameters for a set of given recordings
• extract_surfaces.py - Extract gaze and fixations on surfaces (if available) for a given set of recordings
• extract_blinks.py - Extract blinks for a given set of recordings
• extract_fixations.py - Extract fixations for a given set of recordings
• Fixation Correction - Fixation correction plugin
• batchExportPupilLabs - Batch exports prercorded data as csv for a set of given recordings
• Head pose rotation to yaw/pitch/roll - The Heade Pose Tracker in Pupil Player exports the head pose to head_pose_tacker_poses.csv. The linked function transforms the head pose rotation vector to yaw/pitch/roll in degrees. Starting with Pupil Core v3.4, this feature is built into Pupil Player.
• Post-hoc fixation, saccade, and blink detection - Pipeline that takes in the pupil labs subject data and outputs cleaned fixation, saccades and blinks. It also has a surface detection & mapping option.
• Cognitive workload tool for the Pupil eye tracker - Processing of eye tracking data, using the Pupil Labs eye tracker, for the measure of cognitive workload. Associated thesis.
• CLAHE Scene Video Post-Processing - Preview plugin for Pupil Player and application script to apply CLAHE permanently to the scene video

Small Examples

• Visualize reference data from recorded calibrations
• Example how to read Pupil-Player-generated reference locations

Demo Videos

• Pupil Demo Videos - A playlist of community contributed videos.