SINA is the ros package for Free Space and Object Detection based on deep Learning. This tutorial will introduce included files and necessary steps for starting the program.
- Create a virtual enviroment with virtualenvwrapper. Please install all dependency pacakges from Requirment.txt is in SINA/src
pip install -r requirements.txt- Modify ros path in Semantic.py line 3-4 in order to import 3.4.4 opencv instead ros-opencv-3.3.1 dev
- Modify file path in Semantic.py line 33-35 so that deep learning weights could be imported. The parameter Object_name is to be changed to the traget object name
- Run these commands
roscore
cd catkin_ws/
rosrun SINA Semantic.py
rosbag play -l rgbd_dataset_freiburg1_desk_validation.bag- Here a rosbag recorded by Realsense from TUM is used to test the algorithm, if you want to run the package for your own camera or dataset, please modify the subscriber in Semantic.py line 148
- Result
SINA rospackage contains a whole pipline for training a semantic segmentation netzwork. Relevante files are
- Data Augmentation (automatically).ipynb
- Training for Unet.ipynb
- data
A semantic neural network can be trained by Training for Unet.ipynb after doing data augmentation with Data Augmentation (automatically).ipynb. The data for training will be automatically saved in data file. Please check the path before training.
- The first thing you need to do is to modify the path in Training for Unet.ipynb and Data Augmentation (automatically).ipynb. The data augmentation algorithm based on https://keras.io/preprocessing/image/
-
After finishing the setting up, you can start training Unet
-
To Use the trained Unet, the path of weights in main programm Semantic.py need to be modified
unet = unet_class.Unet('/home/xing_usrqj/catkin_ws/src/SINA/src/fourth_training.h5')- Run the programm
rosrun SINA Semantic.py- The annotation tool for semantic segmentation is “labelme” from MIT. https://github.com/wkentaro/labelme
- A Demo video could be found under SINA/src/Demo