helloitsjoe / webpack-memory-leak

Webpack Memory Leak

This is a basic Webpack setup to reproduce the memory leak described in this Webpack issue: webpack/webpack#17851

Summary

Webpack doesn't fully clean up child compilers from previous compilations in watch mode. HtmlWebpackPlugin creates a child compiler so it's useful in this reproduction, but the leak itself is in Webpack.

How to reproduce

1. yarn
2. Run webpack serve in inspect mode: NODE_OPTIONS=--inspect $(yarn bin webpack) serve
3. Open a memory profiler, for example chrome://inspect in Chrome
4. Take a heap snapshot
5. Save index.template.html, which will cause HtmlWebpackPlugin to create a child compiler. Each time you save, memory should grow ~2mb
6. Take a second heap snapshot and notice the difference in size

Inspecting the diff between heap snapshots you can see a number of object instances are added on each recompilation. Notice these in particular: Compilation, JavascriptParser, FileSystemInfo, NormalModuleFactory, HookMap, Hook. Instances of these objects (and others) should remain constant no matter how many recompilations are triggered. Instead we see new instances added on each recompilation.

Note that you must save index.template.html to see this behavior, since this creates a new child compiler in HtmlWebpackPlugin.

Heap diff before fix

Why aren't these objects garbage collected?

There are a number of WeakMaps used as caches in Webpack and HtmlWebpackPlugin. If we inspect these objects we can see that they always trace back to a WeakMap, for example many of these Compilations are decendents of moduleGraphForModuleMap. We expect 2 Compilation instances at any time: one in _lastCompilation and one as a child of _lastCompilation. These compilations are cleaned up at the beginning of the next compilation, see the source here.

Heap snapshot

moduleGraphForModuleMap

WeakMaps can become a source of a memory leak when another object (including another WeakMap) holds a reference to one of their keys or values.

Heap snapshot

The Fix

Webpack PR here.

In this case I methodically cleaned up WeakMap references in _cleanupLastCompilation until I was able to narrow it down to chunkGraphForChunkMap holding onto RuntimeModules which contain references to compilations from child compilers. The parent compilation reference is cleaned up when _cleanupLastCompilation runs, but the child is still referenced. After calling ChunkGraph.clearChunkGraphForChunk(chunk) on all child compilation chunks the WeakMap references are properly garbage collected.

We're seeing the expected number of instances of Compilation, JavascriptParser, ModuleGraph, etc, and the number of instances stays constant over multiple recompilations. There appears to be a smaller leak in this screenshot but it seems to stabilize over time.

Heap diff before (same as above)	Heap diff after