Js2Hou / Swin-CBIR

This repository contains a CBIR system that uses swin transformer to extract image's feature.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Swin-transformer based CBIR

This repository contains a CBIR(content-based image retrieval) system. Here we use Swin-transformer to extract query image's feature, and retrieve similar ones from image database. Notably, our program achieves intelligent user interaction, including selecting an image by opening explorer dialog and cropping interested region by drafting mouse.

framework

Structure

SWIN_CBIR/
|-- checkpoints/
|
|-- database/
|   |-- data/
|   |   |-- 1.jpg
|   |   |-- 2.jpg
|   |  
|   |-- DB.npz
|   |-- index.txt
|
|-- models/
|   |-- __init__.py
|   |-- build.py
|   |-- swin_transformer.py
|
|-- scripts/
|   |-- generate_DB.sh
|
|-- test/
|
|-- config.py
|-- database.py
|-- generate_DB.py
|-- main.py
|-- requirements.txt
|-- README

Getting Started

  1. Prepare images database

    Just find out some images and put them into database/data/.

  2. Download swin-transformer checkpoint at swin_tiny_patch4_window7_224.pth, and then move it into checkpoints.

  3. run ./script/generate_DB.sh in linux machine to extract features of all images and package them into DB.npz.

  4. run main.py, open an image and select interested region, then program will find similar images in database automatically!

Pay Attention To: we recommend you do step 4 on a local mechine, because it involves graphic user interface.

Results

Here we show two image retrieval results. Two images in the first row are original image and cropped image respectively while the others are retrieval results (have been sorted by similarity).

Note: all images are resize to square for visual requirement, so there would be distorted in some of the images.

frameworkframework

Acknowledgments

Part of code in this repository are copied from Swin-transformer, thank the authors for their exquiste code.

About

This repository contains a CBIR system that uses swin transformer to extract image's feature.


Languages

Language:Python 98.2%Language:Shell 1.8%