JimothyJohn / ojito

A small eye to observe the world

ojito

Ojito (oh-hee-toh), a small eye to observe the world.

Hardware

Ojito utilizes the Seeed Studio XIAO ESP32S3 Sense. Other ESP32 products and cameras could also be used, but memory-restricted (no PSRAM) platforms will probably not work.

Setup

• Create a Replicate API token

• Add your token and Wifi credentials to include/secrets.h

#define REPLICATE_TOKEN "yourtoken"
#define SSID "yourssid"
#define PASSWORD "yourpass"

• Update the host and model version (example) in include/config.h

// Do not add a / to the end of the URL path!
#define HOST "https://api.replicate.com/v1/predictions"
// #define HOST "http://10.0.0.31/predictions"
// Find your version at https://replicate.com/<user>/<model>/versions
#define MODEL_VERSION "0239647164ce56b643213501d63f6116a76a3c620e4194814968e8ba1aa64cb7"

• Update the serial port in platformio.ini:

upload_port = /dev/ttyACM0
monitor_port = /dev/ttyACM0
# Windows ports
# upload_port = COM1
# monitor_port = COM1 
# Mac ports (TBD)
# upload_port = /dev/ttyUSB
# monitor_port = /dev/ttyUSB

• Run the convenience script that installs dependencies, uploads the firmware, then monitors the serial output:

bash utils/Quickstart.sh

Testing

It is highly suggested that you test locally to save money during development!

• Update the host (version not required locally) in include/config.h

// Do not add a / to the end of the URL path!
// #define HOST "https://api.replicate.com/v1/predictions"
#define HOST "http://<your-ip-address-here>/predictions"

• Run a test server locally with:

bash utils/TestServer.sh

Further Reading

• ESP32-S3 Sense
• Cog
• Base64
• ArduinoJson
• PlatformIO

Work to do

• Separate "input" schema from Core library

• Conditionally save images

• Web setup interface

• Find ideal resolution performance

• Add error handling for logner inference times

• Create tests