ealize / reify

Enable ECMAScript 2015 modules in Node today. No caveats. Full stop.

re·i·fy _{verb, transitive}  

re·i·fied _past   re·i·fies _present   re·i·fy·ing _participle   re·i·fi·ca·tion _noun   re·i·fi·er _noun

1. to make (something abstract) more concrete or real
   "these instincts are, in humans, reified as verbal constructs"
2. to regard or treat (an idea, concept, etc.) as if having material existence
3. to enable ECMAScript 2015 modules in any version of Node.js

Usage

1. Run npm install --save reify in your package or app directory. The --save is important because reification only applies to modules in packages that explicitly depend on the reify package.
2. Call require("reify") before importing modules that contain import and export declarations.

You can also easily reify the Node REPL:

% node
> require("reify/repl")
{}
> import { strictEqual } from "assert"
> strictEqual(2 + 2, 5)
AssertionError: 4 === 5
    at repl:1:1
    at REPLServer.defaultEval (repl.js:272:27)
  ...

How it works

Code generated by the reify compiler relies on a simple runtime API that can be explained through a series of examples. While you do not have to write this API by hand, it is designed to be easily human readable and writable, in part because that makes it easier to explain.

I will explain the Module.prototype.import method first, then the Module.prototype.export method after that. Note that this Module is the constructor of the CommonJS module object, and the import and export methods are custom additions to Module.prototype.

module.import(id, setters)

Here we go:

import a, { b, c as d } from "./module";

becomes

// Local symbols are declared as ordinary variables.
let a, b, d;
module.import("./module", {
  // The keys of this object literal are the names of exported symbols.
  // The values are setter functions that take new values and update the
  // local variables.
  default: value => { a = value; },
  b: value => { b = value; },
  c: value => { d = value; },
});

All setter functions are called synchronously before module.import returns, with whatever values are immediately available. However, when there are import cycles, some setter functions may be called again, when the exported values change. Calling these setter functions one or more times is the key to implementing live bindings, as required by the ECMAScript 2015 specification.

While most setter functions only need to know the value of the exported symbol, the name of the symbol is also provided as a second parameter after the value. This parameter becomes important for * imports (and * exports, but we'll get to that a bit later):

import * as utils from "./utils";

becomes

let utils = {};
module.import("./utils", {
  "*": (value, name) => {
    utils[name] = value;
  }
});

The setter function for * imports is called once for each symbol name exported from the "./utils" module. If any individual value happens to change after the call to module.import, the setter function will be called again to update that particular value. This approach ensures that the actual exports object is never exposed to the caller of module.import.

Notice that this compilation strategy works equally well no matter where the import declaration appears:

if (condition) {
  import { a as b } from "./c";
  console.log(b);
}

becomes

if (condition) {
  let b;
  module.import("./c", {
    a: value => { b = value; }
  });
  console.log(b);
}

See WHY_NEST_IMPORTS.md for a much more detailed discussion of why nested import declarations are worthwhile.

module.export(getters)

What about export declarations? One option would be to transform them into CommonJS code that updates the exports object, since interoperability with Node and CommonJS is certainly a goal of this approach.

However, if Module.prototype.import takes a module identifier and a map of setter functions, then it seems natural to have a Module.prototype.export method that registers getter functions. Given these getter functions, whenever module.import(id, ...) is called by a parent module, the getters for the id module will run, updating its module.exports object, so that the module.import method has access to the latest exported values.

The module.export method is called with a single object literal whose keys are exported symbol names and whose values are getter functions for those exported symbols. So, for example,

export const a = "a", b = "b", ...;

becomes

module.export({
  a: () => a,
  b: () => b,
  ...
});
const a = "a", b = "b", ...;

This code registers getter functions for the variables a, b, ..., so that module.import can easily retrieve the latest values of those variables at any time. It's important that we register getter functions rather than storing computed values, so that other modules always can import the newest values.

Export remapping works, too:

let c = 123;
export { c as see }

becomes

module.export({ see: () => c });
let c = 123;

Note that the module.export call is "hoisted" to the top of the block where it appears. This is safe because the getter functions work equally well anywhere in the scope where the exported variable is declared, and important to ensure getters are registered as early as possible.

What about export default declarations? It would be a mistake to defer evaluation of the default expression until later, so wrapping it in a getter function is not exactly what we want.

The important point to understand here is that module.import does not assume a getter function has been registered by module.export for every imported symbol. Instead, parentModule.import only really cares about the contents of childModule.exports. While the childModule.export method helps keep childModule.exports up to date, that level of sophistication isn't strictly necessary in every situation, and default exports are one such situation:

export default getDefault();

simply becomes

exports.default = getDefault();

module.runModuleSetters()

Now, suppose you change the value of an exported local variable after the module has finished loading. Then you need to let the module system know about the update, and that's where module.runModuleSetters comes in. The module system calls this method on your behalf whenever a module finishes loading, but you can also call it manually, or simply let reify generate code that calls module.runModuleSetters for you whenever you assign to an exported local variable.

Calling module.runModuleSetters() with no arguments causes any setters that depend on the current module to be rerun, but only if the value a setter would receive is different from the last value passed to the setter.

If you pass an argument to module.runModuleSetters, the value of that argument will be returned as-is, so that you can easily wrap assignment expressions with calls to module.runModuleSetters:

export let value = 0;
export function increment(by) {
  return value += by;
};

should become

module.export({
  value: () => value,
  increment: () => increment,
});
let value = 0;
function increment(by) {
  return module.runModuleSetters(value += by);
};

Note that module.runModuleSetters(argument) does not actually use argument. However, by having module.runModuleSetters(argument) return argument unmodified, we can run setters immediately after the assignment without interfering with evaluation of the larger expression.

Because module.runModuleSetters runs any setters that have new values, it's also useful for potentially risky expressions that are difficult to analyze statically:

export let value = 0;

function runCommand(command) {
  // This picks up any new values of any exported local variables that may
  // have been modified by eval.
  return module.runModuleSetters(eval(command));
}

runCommand("value = 1234");

exports that are really imports

What about export ... from "./module" declarations? The key insight here is that export declarations with a from "..." clause are really just import declarations that update the exports object instead of updating local variables:

export { a, b as c } from "./module";

becomes

module.import("./module", {
  a: value => { exports.a = value; },
  b: value => { exports.c = value; },
});

This strategy cleanly generalizes to export * from "..." declarations:

export * from "./module";

becomes

module.import("./module", {
  "*": (value, name) => {
    exports[name] = value;
  }
});

While these examples have not covered every possible syntax for import and export declarations, I hope they provide the intuition necessary to imagine how any declaration could be compiled.

When I have some time, I hope to implement a live-compiling text editor to enable experimentation.