This app integrates the Segment Anything Model 1 (SAM) with Sentinel-2 data. The app is built using Dash Plotly and dash leaflet 2. It allows segmenting satellite images using the two ways provided by SAM: automatic mask generator and prompt segmentation (using points and bounding boxes).
- Integration of Sentinel-2 using WMS protocol.
- OSM as base map.
- Downloading Sentinel-2 data directly from the app using an ROI.
- Integration of the three available models from Meta of SAM.
- The ability to directly annotate Sentinel-2 images using bounding boxes and points.
- Different support of bounding boxes:
- ROI
- Object BBox
- Different support of points:
- Foreground points
- Background points
- Modify location of bounding boxes and points.
- Provide prompts for more than one object at a time.
- Refine segmentation inside a bounding box of an object by adding foreground and background points.
- Visualize the obtained results directly on the map.
- Download the results (RGB GeoTiff of the ROI, segmentation mask in PNG and GeoTiff).
- Clone the repository, checkout to the dev branch, create a virtual environment and install the requirements.
git clone https://github.com/AlbughdadiM/satellite-sam-dashboard.git
cd satellite-sam-dashboard
git checkout dev
python3.10 -m venv myven
source myvenv/bin/activate
python3.10 -m pip install -r requirements.txt
mkdir src/weights
wget -P src/weights/ https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
Note that the installation of rasterio requires GDAL to be installed 3. 2. Once all the dependencies are installed, do the following
cd src
python3.10 app.py
- Navigate to 127.0.0.1:8050 to start working with the app.
- Clone the repository, checkout to the main branch, build the docker image and create a container.
git clone https://github.com/AlbughdadiM/satellite-sam-dashboard.git
cd satellite-sam-dashboard
docker build . -t satellite-sam-dashboard:v0.1
docker run --name=dashboard -p 8080:8080 satellite-sam-dashboard:v0.1
Alternatively, you can pull the already-built docker image
docker pull ghcr.io/albughdadim/satellite-sam-dashboard:latest
docker run --name=dashboard -p 8080:8080 satellite-sam-dashboard:latest
The app provides two options to use SAM with Sentinel-2 images:
-
Automatic Mask Generation
- The user draws a bounding box on the ROI to generate a segmentation mask for.
- The bounding box annotation will be added to the table "annotated data" marking xmin, ymin, xmax, ymax, id and the type of the geometry.
- To generate an automatic mask for the whole region covered by the bounding box, the user must select type as "ROI BBox".
- In the Automatic Mask Configuration part of the dashboard, the two parameters prediction IoU threshold and stability score threshold contol the results of the generated mask.
- Once the ROI is selected and the parameters are adjusted, click on Segment ROI.
- The user can visualize the obtained results on the map.
- The user can download the results as a zip file by clicking on Download Results
- To start a new segmentation task, click on Refresh.
-
SAM Prompts
- The user draws a bounding box around the object to be segmented using the bounding box drawing tool.
- The user can obtain a binary mask showing the object inside this bounding box.
- The user can refine the segmentation results by using foreground and background points.
- The user can draw multiple bounding boxes associated with multiple objects in the satellite image. Additionally, each bounding box can be associated with multiple foreground and background points.
- Once the object is selected and the supporting points, click on Segment ROI.
- The user can visualize the obtained results on the map.
- The user can download the results as a zip file by clicking on Download Results
- To start a new segmentation task, click on Refresh.
- On the prod branch, sam_vit_b is the only model that is downloaded to the docker image to keep it lighter. However, feel free to add the other model weights as needed.
- For prompt segmentation, the current code supports two cases: only bounding boxes, and bounding boxes with supporting points. The only points case is not covered as the results were not always satisfactory with Sentinel-2. Note that in the case of bounding boxes and points, if the points are outside the bounding boxes, they will be ignored.
1 https://github.com/facebookresearch/segment-anything/tree/main