yuhogun0908 / Walking-Assistance-System-for-the-Visually-Impaired-in-real-time

Walking Assistance System for the Visually Imparied in real-time

WSVI

WASVI : Walking Assistance System for the Visually Impaired in Real-Time

Methods

[1]. Object Detection : to detect the specific object(person, car, etc.) in each frames. [2]. Multi Object Tracking : to assign and maintain a unique ID to each object in a video. [3]. Traffic Light Detection : to detect the traffic light signal which is various over the country. [4]. Pedestrian Intention Prediction : to make the system possible to predict the surrounding pedestrian intention.

Asuumption

The visually impaired has ability to follow the direction well with his cane on the sidewalk. They don't visit the crowded place. They go to fiamilar, and frequently visited place.

Our Contribution

Make the system possible to predict the surrounding pedestrian movement. Detect the traffic light and consider the signal which is dependent on the country Detect the crosswalk and its direction(not yet)

• Overall Architecture

• Detect Traffic Light Signal

• Detect Pedestrian Intention Prediction

• Detecting CrossWalk

setup python packages environments

pip install -r requirements.txt

DOWNLOAD OBJECT DETECTION MODEL

Download the yolor_p6.pt and save it in the yolormodel folder.

https://drive.google.com/file/d/10ZobLxTBr5r5gDZ1tq3uYn6Sq1YpoU0T/view?usp=sharing
<Reference : https://github.com/WongKinYiu/yolor>

Run

python superman.py --video2frames True --videoName Final1.mp4 --use_cuda 1 

Result

• Detect the signal from the traffic light

• Predict the pedestrian intention and warning

• Detect the crosswalk

Time Break-down

Reference

• [1]. Wang, Chien-Yao, I-Hau Yeh, and Hong-Yuan Mark Liao. "You only learn one representation: Unified network for multiple tasks." arXiv preprint arXiv:2105.04206 (2021).
• [2]. Bouhsain, Smail Ait, Saeed Saadatnejad, and Alexandre Alahi. "Pedestrian intention prediction: A multi-task perspective." arXiv preprint arXiv:2010.10270 (2020).
• [3]. Romic, Kreimir, et al. "Real-time multiresolution crosswalk detection with walk light recognition for the blind." Advances in Electrical and Computer Engineering 18.1 (2018): 11-20.