patVlnta / Ampel-Pilot

Pedestrian Traffic Light Detector for visually impaired people

Ampel-Pilot

Pedestrian Traffic Light Detector for visually impaired people, that can be used as guidance for determining the current phase of a pedestrian traffic light.

You can watch a full demo video here or here. (Shot with a iPhone 6s)

Features

• Detection and recognition of pedestrian traffic lights
• Audiovisual and haptic feedback based on the current traffic light phase (Red, Green)
• Accessibility added for easier usage of the app
• Option to customize feedback and detection parameters

Model and Dataset

The model used in the application is trained with the ML framework YOLOv2. 3062 Images have been used during training given the following results on the validation set (630 images):

Light Phase	Recall	Precision	IoU
Red	0.796	0.739	0.602
Green	0.734	0.688	0.601

Please consider that the model is trained on the Red and Green image channels only. Therefore the camera output is beeing manipulated accordinly before beeing sent further down the proccessing pipeline. You can adjust that via a color matrix here:

VideoCapture.swift

if let colorMatrixFilter = CIFilter(name: "CIColorMatrix") {
            let r:CGFloat = 1
            let g:CGFloat = 1
            let b:CGFloat = 0
            let a:CGFloat = 1

The dataset has been collected in a joint effort between the Hochschule Augsburg and the University of Tuebingen. Students were able to contribute by sending their pictures of pedestrian traffic lights with the LightsCatcher application. Today the datasets consists of around 3600 images and 4300 annotations.

• Dataset repository
• LightsCatcher (Android)
• LightsCatcher (iOS)

Limitations

• Model trained on german traffic lights only
• Using the app at night will get you less accurate results
• Multi lane crossings (3+) will get you less accurate results

Requirements

• Xcode 8 or higher
• iOS 11 or higher

Contributions

PR´s and/or contributions to the dataset are always very welcome. If you have any further questions, ideas or enquiries, feel free to get in contact either by opening an issue or email [e-mail].

Credits

• Inspired by hollance´s YOLO-CoreML-MPSNNGraph
• pjreddie and all contributors for YOLO/darknet
• Project team @Hochschule Augsburg
• Dataset contribution @University of Tuebingen