(potential) memory leak in GAP calculator (ASE) used with quippy
richardjana opened this issue · comments
I have a database of structures stored in extended xyz format, appended in a single file and would like to calculate their GAP energies, forces etc. for all of them using ASE via quippy. Doing this, I run out of memory at some point, as was described in QUIP issue 235. Upgrading f90wrap to the newest version (opposed to the one I got from pip before) led me to the error discussed in issue 137. So I followed the proposed fix there and downgraded my f90wrap version to a580827, which should address the problem. However, then I got the memory leak again...
I checked that I have no other f90wrap versions on my machine, and the fact that I can reproduce the error from issue 137 also indicates that I am using the correct version of f90wrap. Just to be sure, I also checked that the changes in source code were there.
Therefore, it seems to me like there might be another problem in the code, for the way I intended to use it.
Here is a minimal ASE script for my problem:
`
import gc
import numpy as np
from ase import Atoms
from ase.io import read, write
import quippy
gap_calculator = quippy.potential.Potential('IP GAP', param_filename='carbon.xml')
energy_GAP = []
for i in range(1000): # read the first 1000 configurations; this is more than enough to fill my memory
try:
atoms = read('DB.xyz', format = 'extxyz', index = i)
except:
break
n_atoms = atoms.get_global_number_of_atoms()
atoms.calc = gap_calculator
energy_GAP = np.append(energy_GAP, atoms.get_total_energy()/n_atoms)
del atoms
gc.collect()
`
Note that my potential contains TurboGAP descriptors. I don't know if this is relevant, but now you know.
FWIW, I just checked with one of my (2b + 2xSOAP, not TurboSOAP) GAPs, and memory use was constant. It's either a turbo-SOAP issue, or there's something else wrong. I believe I'm using the latest QUIP and GAP, and f90wrap --version
returns 0.2.2.
Also, note that unless you have some other reason to write the code the way you did, it's possible to be a lot more compact, e.g.:
for at in iread('DB.xyz', '0:1000'):
at.calc = gap_calculator
energy_GAP.append(at.get_total_energy()/len(at))
If nothing else, that's definitely going to be a lot more efficient with the I/O (since it won't re-open the file and seek to the correct config 1000 times).
Would you be willing to try turning off TurboSOAP to see if the leak goes away?
Of course! I retrained my potential, keeping everything the same, except that I removed the soap_turbo descriptors. (So now I have only 2b and 3b.) Now I don't seem to have the issue anymore (or rather, the issue only seems to come up when I use soap_turbo descriptors).
@bernstei: Thanks for the tip on reading my input. I will definitely use that!
Thanks, that’s good to know. It could be that without the multi body term the leak is just much slower, so could you try one final test: switch turbo soap for a standard SOAP descriptor?
After recompiling the f90wrap version and quippy/QUIP again, now all potentials (with and without soap_turbo) suddenly seem to work. I am really not sure why, as I thought I had tried exactly this setup before (but apparently this was not the case).
Sorry for causing all this trouble and a big thank you to all who helped!
Edit: I wasn't sure if I was supposed to close this issue just now; I reopened it to enable any possible answers. Let me know when can close it.
No worries, glad this is resolved. I'll close the issue, but do reopen if the memory leak comes back.