reakain / cozmo-arm-nuzzle

Cozmo Arm Nuzzling

Term project for intro to robotics class, focused on developing routines and sensing for the Cozmo desktop robot to recognize arms and hands and appropriately bump or "nuzzle" them as a means of displaying affection.

VIDEO

Environment Setup

This application uses python 3, and these instructions assume python3 is set as your default python command. You can check your version with python --version

1. In bash (or git bash on Windows) navigate to this respository folder, then create your virtual environment with python -m venv ./venv
2. Once complete activate your environment with source ./venv/Scripts/activate
3. Install the requirements once activated with pip install -r requirements.txt
4. If you install any additional libraries, update the requirements with pip3 freeze > requirements.txt
5. Exit the virtual environment with deactivate

State Assumptions

• Cozmo will begin on the charger
• The charger will be placed near the back of the desk, nominally facing the target
• The desk will be nominally free of clutter and without obstacles.
• If a cliff is detected the target cannot be reached.
• If a bump is detected the target has been reached.
• The nudge force can be moderated in some way.
• The environment will be well lit, and the target will be easily separated from the background objects
• The environment will only have one target object (I.e. only one person)
• Cozmo will drive straight in the direction it is pointed.

Machine Vision

We used YOLO to track a person's face.

Bump Sensing

Cozmo lacks a true "bump" sensor, so instead we implemented rough bump sensing using a rolling average of the root mean square of the acceleration. We only track the root mean square during the drive section to avoid tracking logic on the other drive and stop motions.

Control Loop

Obsolete code

Everything in old folder is from initial testing

References

Part 1

1. OpenCV
2. pycozmo
3. cozmo SDK
4. OpenCV tutorials
5. imutils
6. OpenCV feature detection
7. OpenCV edge detection
8. Google hand tracking
9. OpenCV Shape Detection
10. More Shape Detection
11. Object Recognition
12. Contours
13. Object Detection
14. TensorFlow? TensorFlow
15. More TensorFlow

Part 2

1. OpenCV Distance
2. OpenCV Distance between objects
3. Raytracing oh no
4. Finger detection and tracking
5. Distance overlay options
6. Single camera distance estimate
7. cam distance calibration
8. Object localization
9. Camera Calibration in OpenCV
10. IDK size and distance
11. Optical Flow
12. OpenCV real time pose
13. OpenCV py pose
14. OpenCV Dense Optical Flow
15. DeepSort Tracking and Relative Distance