Double Take

Unified API for processing and training images with DeepStack, CompreFace, or Facebox for facial recognition.

Use Cases

Frigate

Subscribe to Frigate's MQTT events topic and process images from the event for analysis.

When a Frigate event is received the API begins to process the snapshot.jpg and latest.jpg images from Frigate's API. These images are passed from the API to the detector(s) specified until a match is found above the defined confidence level. To improve the chances of finding a match, the processing of the images will repeat until the amount of retries is exhausted or a match is found. If a match is found the image is then saved to /.storage/matches/:name.

Known Issues

In rare scenarios, requesting images from Frigate's API causes Frigate to crash. There is an open issue with more information, but it appears sometimes the database connection isn't being closed in time causing Frigate's API to crash. Double Take does use random jitter up to 1 second before all Frigate API requests to help reduce the likelihood of the API crashing.

Home Assistant + Node-Red

Double Take can be paired with Home Assistant and Node-Red to create automations when matching faces are detected.

Other Methods

The API can also be invoked manually for processing. See below for more information.

API

`GET - /recognize`

Process images for recognition with a GET request.

Query Params	Default	Description
url		URL of image to pass to facial recognition detectors
attempts	`1`	Number of attempts before stopping without a match
results	`best`	Options: `best`, `all`
break	`true`	Break attempt loop if a match is found
processing	`parallel`	Options: `parallel`, `serial`
camera	`double-take`	Camera name used in output results
room	`Double Take`	Room name used in output results

Sample Input

/recognize?url=https://your-image.com/sample.jpg

Sample Output

{
  "id": "c16052fe-b110-4d33-b73d-adfee2cf82b8",
  "duration": 1.15,
  "time": "03/16/2021 02:25:21 AM",
  "attempts": 2,
  "camera": "double-take",
  "room": "Double Take",
  "matches": [
    {
      "duration": 1.15,
      "name": "david",
      "confidence": 76.07,
      "attempt": 1,
      "detector": "deepstack",
      "type": "manual"
    }
  ]
}

`GET - /train/add/:name`

Train detectors with images from ./storage/train/:name. Once an image is trained, it will not be reprocessed unless it is removed via the API.

`GET - /train/remove/:name`

Remove all images for the specific name from detectors.

`GET - /train/:camera/:name`

Train detectors with the latest.jpg image from a Frigate camera.

Query Params	Default	Description
attempts	`1`	Number of `latest.jpg` images to use
output	`html`	Options: `html`, `json`

MQTT

If MQTT is enabled and a match is found then a new topic will be created with the default format being double-take/matches/:name.

Sample Topic Value

{
  "id": "1614931108.689332-6uu8kk",
  "duration": 0.85,
  "time": "03/05/2021 02:58:57 AM",
  "attempts": 4,
  "camera": "living-room",
  "room": "Living Room",
  "match": {
    "name": "david",
    "confidence": 82.6,
    "attempt": 1,
    "detector": "compreface",
    "type": "snapshot",
    "duration": 0.39
  }
}

Usage

Basic: Docker Run

docker run -d \
  --name=double-take \
  --restart=unless-stopped \
  -p 3000:3000 \
  -e DETECTORS=facebox \
  -e FRIGATE_URL=http://frigate-url.com \
  -e FACEBOX_URL=http://facebox-url.com \
  jakowenko/double-take

Basic: Docker Compose

version: '3.7'

services:
  double-take:
    container_name: double-take
    image: jakowenko/double-take
    restart: unless-stopped
    environment:
      DETECTORS: facebox
      FRIGATE_URL: http://frigate-url.com
      FACEBOX_URL: http://facebox-url.com
    ports:
      - 3000:3000

Advanced: Docker Compose

version: '3.7'

services:
  double-take:
    container_name: double-take
    image: jakowenko/double-take
    restart: unless-stopped
    volumes:
      - ${PWD}/.storage:/.storage
    environment:
      DETECTORS: compreface, facebox
      MQTT_HOST: mqtt.server.com
      FRIGATE_URL: http://frigate-url.com
      FACEBOX_URL: http://facebox-url.com
      DEEPSTACK_URL: http://deepstack-url.com
      COMPREFACE_URL: http://compreface-url.com
      COMPREFACE_API_KEY: COMPREFACE-API-KEY
      SNAPSHOT_RETRIES: 20
      LATEST_RETRIES: 20
      CONFIDENCE: 65
    ports:
      - 3000:3000

Options

Configurable options that can be passed as environment variables to the Docker container.

Name	Default	Description
DETECTORS		Comma separated list of detectors to process images with: `compreface`, `deepstack`, `facebox`
PORT	`3000`	API port
MQTT_HOST		MQTT host address
MQTT_USERNAME		MQTT username
MQTT_PASSWORD		MQTT password
MQTT_TOPIC	`frigate/events`	MQTT topic for message subscription
MQTT_TOPIC_MATCHES	`double-take/matches`	MQTT topic where matches are published
DEEPSTACK_URL		Base URL for DeepStack API
FACEBOX_URL		Base URL for Facebox API
COMPREFACE_URL		Base URL for CompreFace API
FRIGATE_URL		Base URL for Frigate
FRIGATE_CAMERAS		To only watch specific cameras pass the names in a comma seperated list: `family-room`, `office`, `basement`
FRIGATE_IMAGE_HEIGHT	`800`	Height of image passed for facial recognition
COMPREFACE_API_KEY		API Key for CompreFace collection
SNAPSHOT_RETRIES	`10`	Amount of times API will request a Frigate `snapshot.jpg` for analysis
LATEST_RETRIES	`10`	Amount of times API will request a Frigate `latest.jpg` for analysis
CONFIDENCE	`50`	Minimum confidence level for a face match
SAVE_UNKNOWN	`false`	Save unknown faces to `/.storage/matches/unknown`
PURGE_UNKNOWN	`48`	Hours to keep unknown images until they are deleted
PURGE_MATCHES	`48`	Hours to keep match images until they are deleted
LOGS		Options: `verbose`
TZ	`UTC`	Time zone used in logs
DATE_TIME_FORMAT		Defaults to ISO 8601 format with support for token-based formatting

moandcompany / double-take