okankop / Driver-Anomaly-Detection

PyTorch Implementation of "Driver Anomaly Detection: A Dataset and Contrastive Learning Approach", codes and pretrained models.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

More RAM is consumed as training epoch increases

hongvin opened this issue · comments

I have repeated the experiment for many times and eventually at the 14th epoch, the process is killed, mainly due to my CPU RAM is exhausted. My current configuration is 8 core CPUs, 30GB CPU RAM, P5000 16GB GPU.

I have tried to reduce memory bank size, but it's still the same.

reduce num_worker thread and batch size.

I have the same problem when I try to train this model, and anything can I do?