How to balance the imbalanced classes ?

[Mohammad Amin Dadgar]

Hi, Recently I tried transfer learning methods to this dataset and I got about 70 percent accuracy on the validation set when weighting the classes. When I tried not to weight the classes I got about 80 percent validation accuracy. But when it comes to predicting, The model prediction result is always between 2 classes and never got the other classes in the result. Is there any other method better than weighting my classes? Or any opinion do you have why it is working like that?

The code:

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow import keras
############################# Creating Model #############################
base_model = keras.applications.Xception(
    weights='imagenet',  # Load weights pre-trained on ImageNet.
    input_shape=(150, 150, 3),
    include_top=False)  # Do not include the ImageNet classifier at the top.

image_size=(150,150)

inputs = keras.Input(shape= image_size + (3,))
## Set the base model trainable false to freeze training on the resnet model
# x = base_model(inputs, training = False)
base_model.training = False
model = keras.Sequential()
model.add(base_model)
model.add(keras.layers.Flatten())
## Dense layer with 20 nodes ( we had 20 classes )
model.add(keras.layers.Dense(20, activation='softmax'))

############################# Getting dataset batches #############################
train_generator = tf.keras.preprocessing.image.ImageDataGenerator(
    rescale=1./255,
    validation_split=0.1)
valid_generator = tf.keras.preprocessing.image.ImageDataGenerator(
    rescale=1./255,
    validation_split=0.1)
### Note here we classified folders so we're using flow_from directory method
train_dataset = train_generator.flow_from_directory('dataset',
                                                    target_size=image_size, 
                                                    batch_size=12,
                                                    subset='training',
                                                    class_mode='categorical',
                                                    shuffle=True,
                                                    seed=123)
validation_dataset = valid_generator.flow_from_directory('dataset',
                                                          target_size=image_size,
                                                          batch_size=12,
                                                          shuffle=True,
                                                          subset='validation',
                                                          interpolation='bilinear',
                                                          class_mode='categorical',
                                                          seed=123)

############################# Calculating classes weight #############################
cloth_df = pd.read_csv('images.csv')

## Get the each class values count 
class_values_count = cloth_df.label.value_counts()

## Sort them by index ( indexes are label names as 'T-Shirt', 'Blouse',etc )
class_values_count = class_values_count.sort_index()


## Because our classes don't have similar count
## we would define a function to calculate wights for us
def calculate_weights(class_values_count):
    ## get the max class count
    max_count = class_values_count.max()
    ## Create an empty dictionary
    weights = {}
    ## indexes are used to add to dictionary
    index = 0
    for counts in class_values_count:
        ## Calculate the weights
        ## max_count is 1011 for T-Shirt class
        weights[index] = max_count/counts
        index+=1
    
    return weights
        
class_weight = calculate_weights(class_values_count)
## Also the scikit learn function for weighting was tried ( It is as below )
## sklearn.utils.class_weight.compute_class_weight('balanced', np.unique(values), values)

############################# Compiling and Training the model #############################
model.compile(optimizer=keras.optimizers.Adam(),
            loss="categorical_crossentropy",
            metrics=["categorical_accuracy"],)


epochs = 40

callbacks = [
    tf.keras.callbacks.ModelCheckpoint("transfer_learning_save_at_{epoch}.h5"),
]

## Here I tried both with weight and without weight training
## The last time I tried without weight training, So class_weight is commented
history = model.fit(
    train_dataset, epochs=epochs, callbacks=callbacks,
    validation_data=validation_dataset
#     ,class_weight=class_weight
)