Opportunistic GC collection
maleadt opened this issue · comments
As suggested by @gbaraldi: keep track of GPU memory pressure (using some atomic globals), and when allocating opportunistically call the GC to avoid only reclaiming memory when we run out of it.
Not sure how it impacts your decision here, but I ran into a lack-of-GC issue here (FluxML/Flux.jl#2414) which was completely solved by running GC.gc(false) every other iteration. Running the full GC helped over nothing at all, but was still much slower than the incremental sweep.
Maybe you can try the linked PR? It needs tuning of the heuristic, so datapoints would be useful.