akbartus / MobileSAM-in-the-Browser

Demonstration of MobileSAM in the browser enabled through ONNX runtime web

MobileSAM-in-the-Browser

Description / Rationale

This is the example implementation of MobileSAM in the browser.

Notes

• ONNX runtime web ver. 1.14.0 was used because it was the most compatible with MobileSAM. Unfortunately latest versions on ONNX runtime web did not show good results in segmentation tasks.
• When converting SAM to encoder and decoder some options need to be played with in order to achieve the best results.
• The example uses converted MobileSAM encoder in onnx format. It can be downloaded from here. There are also normal and quantized versions of the decoder, which can be found in "models" folder.

Tech Stack

The project was made possible thanks to SAM, MobileSAM and SAMExporter repositories. In particular, SAM's original model was used in conversion to ONNX and creating a decoder. MobileSAM's original file was used in converting to ONNX and creating encoder. SAMExporter was used in doing conversions to ONNX.

Demo

Demo: Demo