GreenChang01 / SAM-remove-background

Extract objection and remove background application using Segment Anything Model(SAM).

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

SAM Extract Object & Remove background App

Segment Anything Model(SAM) is the foundation model for the segmentation task by Meta. In this repository, an application that extract an object and remove the background by utilizing SAM's interactive segmentation with click is implemented. The UI is implemented using gradio. The Encoder part of SAM is served to the triton inference server to increase the efficiency of inference. All parts of the app were configured for deployment on docker and k8s.

Demo video(Youtube):

SAM.Extract.object.and.Remove.background.App.mov

Prerequisite

  • python 3.9+
  • docker
  • GPU: Required for the speed of the image encoder

Contents

  • [Done] Implement remove background app using SAM with Gradio.
  • [Done] Docker compose for the server and client.
  • [Done] Convert pre-trained SAM Encoder to torchscript. (Huggingface)
  • [Done] Triton serving SAM Encoder.
  • Load test on Triton server (Locust).
  • Kubernetes helm charts for the server and client.
  • Monitoring on K8s (Promtail + Loki & Prometheus & Grafana).

Application Structure

app-structure

SAM has three components: an image encoder, a flexible prompt encoder, and a fast mask decoder. The image embedding obtained by the image encoder, which is a large model, can be reused in the image decoder.

The structure of the application reflects the structure of SAM. The image encoder works on the server part. It uses GPU resources to make inferences. The image encoder is only performed when a new image is uploaded. The relatively lightweight mask decoder and prompt encoder work on the client part. They take the image embedding obtained by the image encoder as input.

Run

1. In local with conda

Initialize conda environment.

make env
conda activate sam-remove-background
make setup

Download models.

make model

Run API Server for SAM encoder.

make server

Run Gradio UI.

make client

Browse localhost:7860.

2. Docker compose

Download models.

make model

Run services with docker-compose.

docker compose up -d

Browse localhost:7860.

Model repository

For this project, I used one of the pre-trained SAM models, the sam_vit_h model. The decoder was converted to onnx and the encoder was converted to torchscript for uploading to triton. Both models were uploaded to huggingface (encoder | decoder).

About

Extract objection and remove background application using Segment Anything Model(SAM).

License:Apache License 2.0


Languages

Language:Python 53.2%Language:Jupyter Notebook 35.7%Language:Makefile 4.4%Language:Shell 3.5%Language:Dockerfile 3.3%