AminAlam / Deep-Object

Joint Deep Neural Networks for Simultaneous Object and Depth Detection

Deep-Object

What is this?

This project implements two concatenated deep neural networks which perform joint object detection and depth estimation. We used Mask R-CNN for object detection network, and MiDaS network for depth estimation. This project was done for EE-SUT deep learning course.

Dependencies

You can install the dependencies by running: pip install -r requirements.txt

Training

Networks Inintial Weights

Both of the networks(object detector and depth estimator) were pretrained on large dataseta. We fine-tuned them using [NYU Depth Dataset V2](NYU Depth Dataset V2) dataset.

Data Augmentatoin

We used following transforms as augmentation:

• Random crop
• Random horizontal flip
• Brightness change
• Random affine rotation (5 to 10 degree)

Loss and Metric Plots

Object Detector

Training

Validation

Depth Estimator

Training

Validation

Joint Object Detection and Depth Estimation BELU

Testing

You can see some test images in the following plots:

Etimated Depth	Detected Objects with Estimated Depth

GUI

You can use the networks by GUI. First, install the depenedncies and the run the follosing command from the project main directory: ptyhon3 GUI/main.py. You can load more images to the app and move between the results using the slider placed at the bottom of the app page. Note that you shoud first train the models and save them in the ./Models directory. For saving or loading the models, change the load_model and save_model boolians in the train.ipynb Configs section.