wrong predictions
AqsaHassan opened this issue · comments
Hey...thanks a lot for resolving the last issue and sorry for delay
The model has successfully trained but unfortunately giving wrong predictions i.e predicting all test images as result : /resnet152/resnet152.pt [[ 5.5552482e+11 4.6270378e+10 -5.7006516e+11]]
ford
OUTPUT:
/home/aqsa/anaconda3/envs/VGG/lib/python3.7/site-packages/sklearn/metrics/classification.py:1437: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no pr**edicted samples.
'precision', 'predicted', average, warn_for)
(VGG) aqsa@aqsa-OptiPlex-7060:~/Car-Model-Recognition-master-try3$ python3 main.py -i hondatest/filehonda2.jpg
input: hondatest/filehonda2.jpg
results/resnet152/resnet152.pt
[[ 1.5563196e+12 5.7977353e+11 -2.1043264e+12]]
ford
input: hondatest/filehonda3.jpg
results/resnet152/resnet152.pt
[[ 1.5563196e+12 5.7977353e+11 -2.1043264e+12]]
ford
input: fileford5.jpg
results/resnet152/resnet152.pt
[[ 5.5552482e+11 4.6270378e+10 -5.7006516e+11]]
ford
input: fileford4.jpg
results/resnet152/resnet152.pt
[[ 5.5552482e+11 4.6270378e+10 -5.7006516e+11]]
ford
Hey, how many epochs did you train?
Configuration
DEBUG = False
training
EPOCHS = 1
MOMENTUM = 0.9
LEARNING_RATE = 0.01
BATCH_SIZE = 16
THREADS = 8
USE_CUDA=False
file paths
IMAGES_PATH = "dataset"
TRAINING_PATH = "train_file.csv"
VALIDATION_PATH = "test_file.csv"
TEST_PATH = "test_file.csv"
RESULTS_PATH="results"
try to put EPOCHS = 10 or at least EPOCHS = 3
okay
More epochs you trainer more the model elaborate your data, I used with a 8 GB NVIDIA GPU to make 25-80 epochs, 3 epochs probably is too low, if you can try with at least 25.
If you still get some problems send the resnet152_graph.png file here, it allows to understand better the state of the model.
If you reach your goal, leave a feedback here and close the issue ;-)
What should I do now? already I have reduced the batch size... Previously I was training on 3 thousand images per class and now I'm training on only 605 images per class.
there's nothing in this graph as well :/
Could you please mention the maximum accuracy you achieved after training ?
again detecting all images as honda_civic
Configuration
DEBUG = False
training
EPOCHS = 25
MOMENTUM = 0.9
LEARNING_RATE = 0.01
BATCH_SIZE = 2
THREADS = 8
USE_CUDA=True
file paths
IMAGES_PATH = "dataset"
TRAINING_PATH = "train_file.csv"
VALIDATION_PATH = "test_file.csv"
TEST_PATH = "test_file.csv"
RESULTS_PATH="results"
Try to increase to 1k or 2k the items inside honda_civic and the same for Ford
and train again
Or if you already have a lot of items, try to increase the epochs also because the graph seems to be still unstable.
Try 50 epochs.
A good graph "stable" could be like this one:
After 45-50 epochs (number bottom of the graph), the blue curve is stable and does not have peaks down.
Also the testing curve (the orange one) is pretty "stable", even with some peaks, for the testing is normal that the peaks are frequently.
this is what I got after 7 hours of training..
Is it possible to train it for at least 45 classes with 86% accuracy? right now I have just tested this project using 3 classes with 50 epochs with following results.
Accuracy: 0.6013851481338977
Confusion Matrix:
[[25209 15663 10078]
[16117 22565 12318]
[ 8328 11482 8190]]
F1 Score:
[0.50115304 0.44811836 0.27958898]
TL;DR; Yes, but you need data
Full answer:
In Machine Learning the problems are:
- DATA
- algorithms behind the neural network
But there are several neural network algorithms available online like resnet152 (that this project use), vggNET etc.
The DATA is the problem in your case, indeed, if you have a lot of data PER class you can do anything with this project, you have to train it a lot (I think that 50 epochs is enough, with more classes just look at the results_graph and try with 80 if you think aren't enough).
Good data should be 10k per class (for example), but I think that also 1-5k is enough to get 60% accuracy.
I made several tests, and the problem was the DATA "labelled", indeed, I only found public data labelled for "vehicle detection" not "vehicle model recognition".
Anyway, you can try to train the model with 80 classes, but it's a big deal 😅.
P.S. = if you solved, we can close this issue 😉
Anyway, thank you, I just added more info about training and epochs parameter in the README for future users/developers 😉
I got good training accuracy but test accuracy is not good as I expected. What does cause bad testing accuracy?
so using the model that you trained on testing you do not get good results?
try to select only balanced data like 1k items per class during the training
@vokhidovhusan have you deal with it? I have the same issue, test accuracy is not growing. I have 5 classes and equal number of images per class
how many images do you have per class?