JinXins/SUMix

SUMix: Mixup with Semantic and Uncertain Information (ECCV 2024)

Huafeng Qin^1,2,5*,†, Xin Jin^1,2,*, Hongyu Zhu^1,2,*, Hongchao Liao^1,2, Mounim A. El-Yacoubi³, Xinbo Gao⁴

¹Chongqing Technology and Business University

²National Research base of Intelligent Manufacturing Service

³Telecom SudParis, Institut Polytechnique de Paris

⁴Chongqing University of Posts and Telecommunications

⁵Chongqing Micro-Vein Intelligent Technology Co.

^* Equal Contribution ^† Corresponding Author

We propose SUMix, which consists of a mix ratio learning module and an uncertain estimation module. The former focuses on computing the proportion of two images and the latter aims to learn the uncertainty information of mixed samples. Firstly, we design a function to compute the semantic distance between the mixed and original samples to determine the ratio lambda. Secondly, we present a method to learn the uncertainty of the mixed. This adapted feature vector effectively mitigates issues related to computing the loss function caused by discrepancies in semantic and uncertainty aspects.

📬 You can contact me by email: 158398730@qq.com or WeChat: xinxinxinxin_j.

If you are interested in palm or finger vein research, please contact us!

🛠 Installation

💥News! ! !💥
2024-07-8: Please Wait for a while, we will release the code and checkpoints. I am busy in AAAI2025 submission, so please wait few days

🔧How to install?🔧
In fact, you can add our python file in OpenMixup.
There, you can see how to use it and the environment required. What you need to do is add or replace our files by folder inside OpenMixup, and then add the function names of the files in the __init__.py file.
You also can download or find other Mixup methods in OpenMixup("https://github.com/Westlake-AI/openmixup")
Thanks contributors: Siyuan Li(@Lupin1998), Zichen Liu(@pon7) and Zedong Wang(@Jacky1128).

Here are the commands to install OpenMixup

conda create -n openmixup python=3.8 pytorch=1.12 cudatoolkit=11.3 torchvision -c pytorch -y
conda activate openmixup
pip install openmim
mim install mmcv-full
git clone https://github.com/Westlake-AI/openmixup.git
cd openmixup
python setup.py develop

Here are the commands to git clone SUMix

git clone https://github.com/JinXins/SUMix.git

📊 Experiments

CIFAR-100

Name	alpha	Conference	ResNet18	ResNeXt50	Wide-ResNet28	DeiT-Small	Swin-Tiny
CutMix	0.2	ICCV2019	78.17	78.32	84.45	74.12	80.64
SaliencyMix	0.2	ICLR2021	79.12	78.77	84.35	69.78	80.40
FMix	0.2	ArXiv	79.69	79.02	84.21	70.41	80.72
ResizeMix	1.0	CVMJ2023	80.01	80.35	84.87	68.45	80.16
CutMix+SUMix	0.2	-	79.78	79.91	84.56	75.26	80.32
SaliencyMix+SUMix	0.2	-	79.91	79.32	84.58	70.32	84.71
FMix+SUMix	0.2	-	80.20	80.79	84.32	70.69	80.73
ResizeMix+SUMix	1.0	-	80.38	80.72	84.91	68.78	80.59

Tiny-ImageNet & ImageNet-1K(denote *)

Name	alpha	Conference	ResNet18	ResNeXt50	ResNet18
CutMix	0.2	ICCV2019	65.53	66.47	68.95
SaliencyMix	0.2	ICLR2021	64.60	66.55	69.16
FMix	0.2	ArXiv	63.47	65.08	69.96
ResizeMix	1.0	CVMJ2023	63.74	65.87	69.50
CutMix+SUMix	0.2	-	65.71	68.74	69.71
SaliencyMix+SUMix	0.2	-	65.68	68.92	69.52
FMix+SUMix	0.2	-	63.69	67.12	70.48
ResizeMix+SUMix	1.0	-	65.30	67.49	69.76

CUB-200, FGVC-Aircraft and Standford Cars

Name	alpha	Confrence	CUB R18	CUB RX50	FGVC R18	FGVC RX50
CutMix	0.2	ICCV2019	77.70	83.67	78.84	84.55
SaliencyMix	0.2	ICLR2021	75.77	82.83	79.78	84.31
FMix	0.2	ArXiv	77.28	84.06	79.36	84.10
ResizeMix	1.0	CVMJ2023	78.50	84.16	78.10	84.08
CutMix+SUMix	0.2	-	78.20	83.71	79.72	85.84
SaliencyMix+SUMix	0.2	-	76.98	82.84	79.90	84.49
FMix+SUMix	0.2	-	79.24	84.33	79.48	84.64
ResizeMix+SUMix	1.0	-	78.56	84.23	80.29	85.12

😉 Citation

If you feel that our work has contributed to your research, please cite it, 🥰 and please don`t forget to cite OpenMixup if you use this project ! 🤗 Thanks.

@article{qin2024sumix,
  title={SUMix: Mixup with Semantic and Uncertain Information},
  author={Qin, Huafeng and Jin, Xin and Zhu, Hongyu and Liao, Hongchao and El-Yacoubi, Moun{\^\i}m A and Gao, Xinbo},
  journal={arXiv preprint arXiv:2407.07805},
  year={2024}
}

JinXins / SUMix