geohot / body_loop

comma body does a loop around the office

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

comma body loop driver

Can drive the body in the comma office in loops.

It uses a YOLOv8 foundation model to preprocess the images, then a policy model to determine left, right, or straight. The policy model is currently trained with hand labelled data (included), but can be extended with RL.

Preprocessing, training, and inference (both on PC and device) are all done with tinygrad.


We collected a dataset of carrying the body through the loop.

"left" means it's too far left and should go right "right" means it's too far right and should go left "straight" means you are good to go straight "reverse" means you are against a wall and should reverse

Training the model

# First, process the data:

# Then, train the model (tinynet.safetensors):


You can either run on PC remotely controlling a body, or on a comma 3X.



A few standard Python deps. opencv-python is only needed on device.

pip install av tqdm opencv-python

You need a recent version of tinygrad for everything. openpilot for the controls stuff, and cv2 on device for color transform and resizing (TODO: replace with tinygrad!)


comma body does a loop around the office

License:MIT License


Language:Python 99.0%Language:Shell 1.0%