java.lang.OutOfMemoryError: GC overhead limit exceeded

[Gh0st01]

Hello,
First of all, thanks for your awesome work.

I'm facing a strange issue.

When i train my algo with this command :
~/myfolder/StackNet $ java -jar stacknet.jar train task=classification sparse=false, s_head=false, model=/home/user/mymodelfolder/model_final pred_file=/home/user/myfolder/StackNet/apred.csv train_file=/home/user/myfolder/StackNet/trainalgo.csv test_file=/home/ghost/myfolder/StackNet/testalgo.csv test_target=true, params=/home/ghost/myfolder/StackNet/aparams.txt verbose=true threads=7 metric=auc stackdata=false, seed=2 folds=5

everything works, train and testing on my testfile but when i want to test my trained model with this command:
~/myfolder/StackNet $ java -jar stacknet.jar predict sparse=false, has_head=false, model=/home/user/mymodelfolder/model_final pred_file=/home/user/myfolder/StackNet/apred.csv test_file=/home/user/myfolder/StackNet/testalgo.csv test_target=true verbose=true metric=auc

I got this error :

parameter name : sparse value :  false
parameter name : has_head value :  false
parameter name : model value :  /home/user/mymodelfolder/model_final
parameter name : pred_file value :  /home/user/myfolder/stacknet/apred.csv
parameter name : test_file value :  /home/user/myfolder/stacknet/testalgo.csv
parameter name : test_target value :  true
parameter name : verbose value :  true
parameter name : metric value :  auc
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:1079)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2048)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:428)
	at java.util.ArrayList.readObject(ArrayList.java:797)
	at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2173)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2064)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2282)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2206)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2064)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1970)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1562)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2282)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2206)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2064)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1970)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1562)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2282)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2206)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2064)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1568)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1970)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1562)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1970)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1562)

[Marios Michailidis]

Can you try adding this to your java command ? java -Xmx6048m -jar . This tells it to use 6 GB of RAM. You can add more if you want. It could be that due to memory the process becomes mush slower (trying to allocate memory) . See if problem persists with this,

[Gh0st01]

Thank you it worked like a charm !

I don't really understand why since the process was using only 4GB ! But i used 8GB with your arg and it worked in 12sec ! I'll edit my original post to remove useless informations.

Thanks !