2022 인공지능 온라인 경진 대회

Task: 소고기 이미지를 통한 등급 분류 문제

Team Name: SKKUAI (Members: Chaeeun Ryu, Sangeun Lee, Sangmin Han, Jaemin Yoo)

Final Ranking: 16th (top 17%)

과제 설명

✔ 소고기 도축 이미지를 보고 등급을 분류하는 문제

추진배경

✔ 우리나라는 소고기 마블링 스코어를 품질 등급 분류에 주요 변수로 삼기에 외국의 등급 체계와 차이가 있어 수입 소고기에 대해 국내 소비자들의 혼란이 있을 수 있음

✔ 축산 공공 데이터의 활용 방안 확대

Data Preprocessing

Cutmix
Mixup
Combinations of augmentation methods (kornia & pytorch)

Note: Although Cutmix led to considerable enhancement in model's performace, combinations of augmentations didn't always lead to increase in test accuracy. Also, as it was easier to insert code in training rather than making an independent dataset with cutmix or mixup method, we did not make independent dataset for those two methods.

Augmentation method	links
Horizontal Flip + Color Jitter	link
Grayscale + Vertical Flip	link
Center Crop + Color Jitter + Random Erase	link
Horizontal Flip + Random Rotation	link
Random Vertical Flip + Rotation	link
Random Vertical Flip + Center Crop	link
Random Vertical Flip + Color Jitter	link
Cutmix	link
Mixup	link

Note: Currently, the links to datasets are not allowed due to the policy of competition.

Modeling

AlexNet
Sharpest Aware Minimization (SAM)
EfficientNet
ResNet Series
WideResNet
Contrastive Learning (simclr pretrained weights)
Consideration of Color Histogram

Results in numbers

Evaluation Metrics: Weighted Kappa Score

Methodology	Test Accuracy
WideResNet+SAM	0.218
RseNet101	0.865
EfficientNet	0.914
SpiralNet-ResNet	0.905
ResNet34+Endsemble	0.727
Transformer (Backbone: ResNet50)	0.906
WideResNet+Ensemble	0.942
EfficientNet with augmentation	0.930
EfficientNet + Augmentation + Cutmix	0.94
EfficientNet + cutmix	0.932
EfficientNet + Augmentation + Cutmix + lr=1e-03, epoch = 150	0.957
AlexNet+Semi-supervised	0.953
AlexNet+Cutmix+Mixup	0.936
EfficientNet+cutmix+mixup+color histogram (w/o regularization)	0.943
EfficientNet+all augmented data	0.952
Ensemble+augmentation	0.913
Snapshot Ensemble	0.931

Insight: Not all models' performance was improved by large augmented data. And all models faced overfitting, with training accuracy reaching 100%, and yet validation accuracy converging below 100%.

Result Analyis in Visualization

Note: Every accuracy or error rate shown below are based on standard accuracy, and we noted that when we submit our submission file, and assess our submission based on weighted Kappa Score, which takes the relative distance between labels into consideration, the accuracy is calculated higher than ours.

EfficientNet+Augmentated Data + Cutmix + Mixup + Color Histogram