Proposal: Memoization Utility

[Curran Kelleher]

I would like to propose a utility for memoization that works well in conjunction with D3. The purpose of this is for performance optimization in contexts where a function that uses D3 for DOM manipulation is repeatedly invoked as state changes as a result of user interaction.

I've come to use the "unidirectional data flow" pattern quite a lot in my work. With this pattern, a rendering function is invoked over and over again each time some state changes. This works great on its own, and also works great in conjunction with React (putting the D3 logic inside a useEffect hook and letting React manage the state via useState or something else).

In the React world, when the need arises for performance optimization, I would reach for useMemo. However, when developing purely with D3, there is no corresponding D3 API for addressing the same need.

In order to make the same kind of performance optimizations in D3, it might make sense to introduce an API similar to useMemo. It might make sense to store the memoized value and previous dependencies on the DOM (similar to how the brush or zoom state is stored on the DOM) so that user code doesn't need to manage a global store of memoized values somewhere (which I guess is what React does internally).

Does anyone else face this kind of problem? Would such a utility be of interest? Thanks!

[Curran Kelleher]

Strawman API proposal:

const filteredData = d3.memoize(

  // The first argument is the function to be memoized, just like React's useMemo.
  () => data.filter( ... some logic that uses filterState ... ),
  
  // The second argument is the array of dependencies, just like React's useMemo.
  [data, filterState],
  
  // The DOM node to store the memoized value on
  selection.node(),
  
  // A name for this, optional, to support multiple memoize calls on the same DOM node
  'filteredData'
);

[Vijith Assar]

This kind of problem can also be solved by attaching a memoized function to the node using d3.local(), is that insufficient?

[Curran Kelleher]

That would work, but not in the context of hot reloading (d3.local uses a new ID on each run).

Here's a strawman implementation that works under hot reloading as well:

// Akin to React's useMemo, for use in D3-based rendering.
// Memoizes only one computed value.
// Accepts a function fn and dependencies, like useMemo.
// Also accepts a DOM node and a name for the memoization.
// The memoized value and previous dependencies are stored
// on the DOM node, using the given name.
const memoize = (fn, dependencies, domNode, name) => {
  const property = `__memoized-${name}`;
  const memoized = domNode[property];
  if (
    memoized &&
    dependencies.length === memoized.dependencies.length
  ) {
    let dependenciesChanged = false;
    for (let i = 0; i < dependencies.length; i++) { 
      if (dependencies[i] !== memoized.dependencies[i]) { 
        dependenciesChanged = true;     
        break;
      }
    }
    if (!dependenciesChanged) {
      return memoized.value;
    }
  }

  // If we got here, either it's the first run,
  // or dependencies have changed.
  const value = fn();
  domNode[property] = { dependencies, value };
  return value;
};