A random collection of useful embodied AI agent code I wrote, and actively use in my projects.
It's not a complete project, but it should be easy to port this over into your own, locally running realtime agent systems.
lib
contains the main interfaces used to communicate with the backend, as well as the realtime perception driver.electron.cjs
contains the core electron driver which reads the screen.preload.js
contains the web environment hook injected into the browser context.clients.js
contains memory, perception, and other interface code.
The system calls out to these AIs running remotely:
- https://github.com/avaer/sam-blip2
- for captioning/segmentation
- https://github.com/avaer/doctr
- for on-screen OCR