SA-ICM (IEEE ICIP 2024)
This is the official implementation of the following paper.
・Image Coding for Machines with Edge Information Learning Using Segment Anything
(arXiv)
- Clone this repository from GitHub.
git clone https://github.com/final-0/SA-ICM.git
- Change the current directory to the "SA-ICM" folder.
cd SA-ICM/
- Install the required dependencies in the current directory.
pip3 install -r requirements.txt
For testing only, a GPU with about 11GB of memory is sufficient. (e.g. 1080ti, 2080ti)
Download model weights.
You can obtain "icm_78.pth.tar" and "icm_93.pth.tar" from this link.
These weights can be used by placing them in the "param" folder.
param ---- param_details.txt
|-- icm_78.pth.tar
|-- icm_93.pth.tar
If you want to compress images for "Machines" with , run the following command :
python3 coding_m.py --checkpoint param/icm_78.pth.tar --input image/input
python3 coding_m.py --checkpoint param/icm_93.pth.tar --input image/input
Add “--real” to the command to obtain a bit-stream:
python3 coding_m.py --checkpoint param/icm_78.pth.tar --input image/input --real
python3 coding_m.py --checkpoint param/icm_93.pth.tar --input image/input --real
Compressed images will be saved in the folder "output_icm_78" or "output_icm_93".
image ---- input
|-- output_icm_78
|-- output_icm_93
object detection (yolov5)
Download the model weights trained with the compressed images.
You can obtain "yolov5_78.pt" and "yolov5_93.pt" from this link.
These weights can be used by placing them in the "param" folder.
param ---- param_details.txt
|-- icm_78.pth.tar
|-- icm_93.pth.tar
|-- yolov5_78.pt
|-- yolov5_93.pt
The following commands can then be used to detect objects in compressed images.
python3 yolov5/detect.py --weights param/yolov5_78.pt --source image/output_icm_78
python3 yolov5/detect.py --weights param/yolov5_93.pt --source image/output_icm_93
Detection results will be saved in "yolov5/runs/detect/".
* We cite the yolov5 implementation from ultralytics.