depthai TopFormer Semantic Segmentation
Python scripts performing on devive semantic segmentation (ADE20K 150 classes)using the TopFormer model in depthai.
Taken from: https://youtu.be/fzZEylhZTbI
Requirements
- Check the requirements.txt file.
- Additionally, pafy and youtube-dl are required for youtube video inference.
Installation
git clone https://github.com/ibaiGorordo/depthai-TopFormer-Semantic-Segmentation.git
cd depthai-TopFormer-Semantic-Segmentation
pip install -r requirements.txt
For youtube video inference
pip install youtube_dl
pip install git+https://github.com/zizo-pro/pafy@b8976f22c19e4ab5515cacbfae0a3970370c102b
MyriadX models
The original Pytorch models were converted to MyriadX blob using the Google Colab notebook below. You can download the converted models from Google Drive and save them into the modes folder.
- Conversion notebook: TopFormer_to_MyriadX_Conversion.ipynb
- The License of the models is Apache-2.0 License: https://github.com/hustvl/TopFormer/blob/main/LICENSE
ONNX Inference
The model can also run in ONNX using the scripts in this repository.
Pytorch model
The original Pytorch model can be found in this repository: https://github.com/hustvl/TopFormer
Examples
- Depthai Semantic Segmentation using the camera in the board (~20 fps for small model):
python main.py
- Video inference: https://youtu.be/x3UVNlPlFlc
python main_video.py
Original video: https://youtu.be/fzZEylhZTbI
References:
- TopFormer model: https://github.com/hustvl/TopFormer
- ONNX TopForfmer: https://github.com/ibaiGorordo/ONNX-TopFormer-Semantic-Segmentation
- Luxonis Depthai: https://docs.luxonis.com/en/latest/
- PINTO0309's model zoo: https://github.com/PINTO0309/PINTO_model_zoo
- Original paper: https://arxiv.org/abs/2204.05525