Enhanced Automated Quality Assessment Network for Interactive Building Segmentation in High-Resolution Remote Sensing Imagery
Paper Address
This study introduces the IBS-AQSNet, a network designed for interactive building segmentation quality assessment (SQA) in high-resolution remote sensing images. The proposed IBS-AQSNet integrates a powerful pretrained backbone network with a lightweight correspondence network, facilitating efficient and comprehensive feature extraction. This network employs a direct fusion approach for these features and incorporates a multi-scale differential quality assessment decoder, adept at identifying errors in SQA results, including missed and mistaken areas.
Fig.1. The flowchart of the proposed IBS-AQSNet
The proposed method effectively addresses the challenge of building segmentation quality assessment. Experimental validation on the newly built EVLab-BGZ dataset confirms that the IBS-AQSNet excels in the building SQA task, representing a progression in remote sensing image analysis.
Fig.2. IBS-AQSNet performance visualization on EVLab-BGZ under different experimental setups. (a) images & prompts, (b) interactive segmentation results,
(c) SQA ground truth, (d) Baseline, (e) Baseline with PIF, (f) Baseline with PIF and AQSD.Key:PIF (using pre-trained image features from SAM-b’s backbone);
Red for mistaken areas, Green for missed areas.
- EVLab-BGZ dataset: EVLab-BGZ (Extraction code:EVLa)
This dataset comprises 2,825 aerial images and 1,006 satellite images, each with a resolution of 512×512 pixels. It encompasses a total of 39,198 instances of buildings. Each building is individually annotated, with rare instances of multiple buildings annotated together. The dataset features significant variation in building density, ranging from a single building to as many as 51 buildings per image, with each building occupying over 2,500 pixels. The dataset is divided into two subsets: 3,000 images for training and 831 images for testing.
There's still a bit of a problem with the display here, it can be downloaded and viewed, see folder: figures/demo.mp4
The video demonstrates the following steps:
a. Interactive Segmentation of a Building Target: A process where the user directly interacts with the software to segment a building.
b. Heuristic Automatic Detection of Buildings: An automated method using heuristic techniques to identify buildings in an image.
c. Automatic Quality Assessment of Interactive Building Segmentation:
An automated process that evaluates the quality of the segmentation performed in step 1.
- Note: Each interactive building segmentation result is given 2 seconds for automatic quality assessment,
to dynamically display the automatic quality control effect.
d. Manual Interaction to Modify Certain Results: The user manually adjusts or corrects the segmentation results where necessary.