PersonFollowingRobot
Person Following Robot - Smart Trolley utilizing Artificial Intelligence (AI) and 3D vision
This project will introduce a Person Following Robot can follow a person on various complex floors in indoor environments under many challenging: illuminating conditions, pose changes, occlusions, transiently losing the target, the target being covered, etc...
Applied techniques:
- Deep Learning: Object Detection and Single Object Tracking with Vision Transformer
- 3D vision, point cloud processing, depth image (Realsense camera)
- Implement in ROS platform
Hardward specs
- Battery: 18650, 18 cells, 24V 7800mAh
- Voltage Regulator Circuit: 5V-5A
- Motor Driver: Planet 24VDC 60W 600RPM
- H-Bridge Driver
- Camera: Intel RealSense Depth D455
- Microcontroller: STM32F103C8T6
- Board Computer: Jetson Nano
Recognizing floor
The 3D vision algorithms can recognize various types of floors and in different lighting conditions.
Detection and tracking people
We used a pre-trained model available in jetson-inference to detect people. Stark is a single object tracking model using Vision Transformer that we experenced on my laptop to track one person in the frame in different complex cases. All models were converted to TensorRT to accelerate inference speed.
ROS system
Result
The robot aims to operate steadily and is user-friendly.
result.mp4
Authors
To further information, feel free to contact me (Huy An) via Zalo 0773891493 or my team via email:
Nguyen Huy An: anhuynguyen001@gmail.com - AI and Computer Vision
Ngo Tan Hau: ngotanhauios@gmail.com - Hardware and Microcontrol
Pham Trung Nghia: nghiatrung333@gmail.com - Mechanical