Problems classifying gestures with ANBC. Does not classifiy. Output class = 0
alberto-tellaeche opened this issue · comments
Hello all;
First of all apologies if this is not the correct place to pose a question, I have not located any forum or better place to do so....
I have recently dicovered this toolkit. It is just what I need for my developments, however, I am facing a lot of problems with ANBC classifier.
I am using a Kinect to detect static gestures. I build 15 dimension vectors (left and right side of the upper body with respect to torso, using OpenNi + NiTE)
I am trying to detect 4 gestures and my data set is as follows
===============================================================
DatasetInfo:
Number of Dimensions: 15
Number of Samples: 200
Number of Classes: 4
ClassStats:
ClassLabel: 1 Number of Samples: 50 ClassName: GestoStopIzda
ClassLabel: 2 Number of Samples: 50 ClassName: GestoResumeIzda
ClassLabel: 3 Number of Samples: 50 ClassName: GestoStopDcha
ClassLabel: 4 Number of Samples: 50 ClassName: GestoResumeDcha
Dataset Ranges:
[1] Min: -275.564 Max: 400.952
[2] Min: -384.954 Max: 37.4829
[3] Min: -14.7864 Max: 160.272
[4] Min: -95.6251 Max: 34.6766
[5] Min: -271.794 Max: -95.1042
[6] Min: -244.351 Max: -62.3318
[7] Min: -192.111 Max: 199.64
[8] Min: -271.794 Max: -103.72
[9] Min: -197.489 Max: -12.4874
[10] Min: -8.42172 Max: 6.09038
[11] Min: -229.119 Max: -220.108
[12] Min: -11.0981 Max: 6.2278
[13] Min: -8.42189 Max: 6.09359
[14] Min: -453.431 Max: -433.639
[15] Min: -11.0986 Max: 6.22808
[TRAINING ANBC] Training Naive Bayes model, num training examples: 200, num validation examples: 0, num classes: 4
[TRAINING ANBC] Training set accuracy: 99
-244121 -40098.9 -91058.5 -82286.7 (Null Rejection Thresholds. Is this normal ????)
==========================================================================
When splitting the data set in training and testing data as in examples, it gives 98% correct outputs. However, with real new data, I does not classify anything, giving always an output "0"
The classifier has been trained withthe following options:
anbc.enableNullRejection(false);
anbc.setNullRejectionCoeff(10); // Several high values trained here up to 1000
anbc.enableScaling(true);
if(!anbc.train(dataset))
{
cout << "Failed to train ANBC classifier!\n";
return EXIT_FAILURE;
}
Could anyone give me a clue, it is rather important for my job.
Thank you all in advance,
Alberto
Hi Alberto,
Can you give me some info on what types of gestures you are trying to detect with ANBC and the Kinect?
ANBC works well for static postures (like detecting if your arms are down by your side, or if one arm is held up in a pointing gesture, or if both hands are up are your face making a binoculars gesture, etc.), but it doesn't work well for dynamic gestures that have an explicit time component (like swipes, or detecting a circle being drawn in one direction vs. a circle being drawn in the opposite direction).
Are you trying to use it for static postures or dynamic gestures?
Hi Nick;
Thank you for the response. My gestures are static, two "stop" gestures (right and left arm) and two "resume" gestures (right and left arm). The stop gesture is like the "psi" pose of the Kinect, with only one of the arms either left arm (gesture 1) or right arm (gesture 2), simmilar to what traffic policemen do when they stop traffic. The other gesture is the arm extended and raised laterally with respect to the body trunck. Again this can be done either with left arm (gesture 3) or right arm (gesture 4).
The data vector is composed by the (X,Y,Z) positions of the arm joints with respect to the torso in mm. For example for the left arm gestures, these are the transformations I store in the vector:
left hand with respecto to torso -> 3 first values of the vector
left elbow with respect to torso -> 4 to 6th values of the vector
left shoulder with respecto to torso -> ...
neck with respect to torso -> ...
head with respecto to torso -> 12 to 15th values of the vector
The same with the right side of the body. I have recorded 50 instances of each of the 4 gestures. I can provide them to you if necessary.
THe thing is that testing the classifier using cross validation, I works perfect !!, but with new recorded gestures in real time, I get not classification, only the "0" class. and "inf" in all the class distances...
I would appreciate any kind of help, thank you in advance,
Alberto
OK, so ANBC should be great at these kind of gestures with the kinect.
It sounds like you are already translating the data relative to the user's torso, if so this is the right preprocessing step.
If you are doing this, then the main issue sounds like you either have bad samples in your training dataset (e.g., you accidentally pressed record, walked over to perform the gesture, then walked back and pressed stop, giving you lots of bad data either side of your gesture data), or you have not enough variability in your training dataset (i.e., the model has learned very tight sigmas over your gesture data).
If you want to share the training data file with me I can quickly check to see if I see anything obvious in it.
Hi Nick, thanks again.
yes, everything related to the user torso.
About the samples, they are not bad samples, I think I have taken care of it (I press record and wait 5 seconds to start recording N samples, then the recorder stops, so every gesture is recorded correctlt). About the overfitting of the algorithm, it may be. I do not know how to check it.
The dataset is the merge of the four datasets it provide to you (GRT files renamed as TXT to be able to attach them). Also the GRT of the ANBC (ANBCmodel.txt)
I also attach the results of the model training (training model.txt), and also a log file (log.txt), with real time entries vectors constructed with the left body side. In the log file, you can see the vectors, the distance to classes as "inf" and the default output to 0. All this gestures in the log should correspond to "GestoStopIzda" in the dataset.
Really, I would appreciate a lot a hint about all this. This is part of a job for my company and I really like your library. For sure it can be done, but I con not see the error...
Thank you,
Alberto
ANBCmodel.txt
GestoResumeDcha.txt
GestoResumeIzda.txt
GestoStopDcha.txt
GestoStopIzda.txt
log.txt
training model.txt
Thanks for sharing the files, I'll take a look and see if I can spot anything obvious.
In the mean time, you might want to quickly try re-recording the data, but this time when you perform each gesture deliberately move your arms a little for each posture, this will stop the ANBC algorithm learning sigmas that are fit too tightly to your training data, which should help it generalize to new data when you run the live prediction.
Hi Nick,
I am out of the office for a few days.
I already did slight movements while recording, exactly to avoid the problem you mention. You will see it on the data.
Anyway, I will récord new datasets as soon as posible.
Thank you,
Best,
Alberto
Hi Nick,
Hace you had the opportunity to have a look at the data?
I continue trying and honestly don't see where the problem could be...
Thank you,
Alberto
Hello all;
Regarding this issue, does anyone have alternative ideas? I think the procedure I have followed is ok, but I am still not having good results.
Has anyone checked the data published? If any of you has seen something weird in data, please let me know to find a solution.
Thank you all in advance,
Best
Alberto
Hi Alberto,
I've had a look at the data and models.
One thing that pops out is the large variance in the rejection thresholds for each class. The thresholds for each class are listed below:
[1] -86.1614
[2] 8.57917
[3] -21.1061
[4] -18.3597
This indicates that there is a big difference in how classes 1 and 2 are being performed (as class 1 has a very low rejection threshold of -86 and class 2 has a very very high rejection threshold of 8). These thresholds are in log scale, so the difference between -86 and +8 is huge.
Can you try to quickly re-train the model and take these two classes out and see if the model can accurately recognize classes 3 and 4.
Also, I noticed you only have 50 samples each per class. I would recommend recording a lot more training data as this should help build a more robust model. Specifically, I recommend making a number of short recordings for each class (as opposed to one long recording), as this should capture the natural variability of how you are performing each gesture so the classifier provides better predictions for new data.
Hi Nick;
After a while I have managed to get to this problem again. Just writting to you with good news.
I have recorded a a new complete dataset of 300 instances for each class (6 times more) trying to generalize, and now it works quite ok... before it seems I had an overfitting problem...
Thanks a lot for the help, and also for your library, it is awesome !! :-)