This repo contains my solution for the Robotic Instrument Segmentation Sub-Challenge which was part of the Endoscopic Vision Challenge held in conjunction with MICCAI 2017. The challenge webpage can be found here
Fig 1. An illustration of the binary segmentation task segmentation.
In the contest, I implemented a "patch and stitch" CNN strategy followed by CRF post processing.
Essentially, a VGG-19 is trained on 51x51 patches from the input image. The patches containing any part of the medical instrument are labelled foreground and the rest background. Thus, the CNN acts like a foreground detector and is trained to learn a rough contour of the medical instrument.
A CRF is applied to the coarse output of the CNN. I use the CNN output as the unary potential. A binary potential is also applied. This basically 'cleans' up the image by identifying edges of the instrument which are characterised by large colour differences.
To run this code, you require the following softwares.
- Anaconda for python 2.7 - Follow instructions here.
- Pytorch - Get here.
- To get Pytorch to work on GPU, you need to have CUDA.
A pre-trained foreground detector VGG-19 model can be downloaded from here. This model has been trained on 8 video sequences of 225 frames each, that were part of the challenge dataset.
To run the demo
- Download and place the trained model in
Code/models. - Follow the code within
Code/test_patch_cnn_and_crf.ipynbto see the CNN and CRF results.
For any assistance with the code or for reporting errors, please get in touch at rahulduggal2608 [at] gmail [dot] com.
I would like to thank the authors of the following repos/blogs -
- For implementations of standard CNN models - this.
- For the CRF implementation - this.
- Good explanation of CNN+CRFs - this.
This code is released under the MIT License (refer to the LICENSE file for details).