Refactor Language Server Compilation Flow

[Tim Whiting]

Create a separate compiler flow for the language server which:

• Separates import graph resolution from compilation
• Separates compilation into distinct modular phases with timestamps for invalidation
  • Parsing
  • Type Checking
  • Compile
• Creates a separate import graph for each file in the workspace (actually add to Module)
• Keep track of states of modules better: Source / Source Parsed / Source Checked / Source Generated / Source Error / Iface Loaded...?
• Move the parts of Loaded that are really per Module into Module (at least for the Language Server data model)

Essentially move to the something similar to the following data model:

ModuleSource(parseTime, program, errors, importGraph)
ModuleTyped(parseTime, checkTime, program, core, errors, warnings, importGraph, gamma, etc)
ModuleCompiled(parseTime, checkTime, compiledTime, outputs, ...)

Compiled modules should keep track of the flags their outputs are generated under, (i.e. a Map<Flags, OutputLocation>), probably just a subset of the flags - optimization, target, output directory. This way if we want to recompile under another target / optimization we can reuse the typed module part, but keep track of what we have already done, and just add another output to the map.

We should parse & type check all files in the workspace regardless whether they are opened or not (in order to report diagnostics / highlight the files red). However, we should always prioritize the files that are open.

Parallelization should be possible if we are smart and take into account a full workspace import graph, and then sort it by priority files, and mark available files when their dependencies are done (at least for type check / compile, parsing can be done fully in parallel, and building import graphs should be done smartly).

Additionally as we do this, make sure there is no memory leaks between stages. (i.e. make sure everything in the Module is fully evaluated).

When a source is changed (either due to being updated in the virtual file system, or on the file system itself - we should add a watcher), we need to invalidate appropriately, and don't try recompiling all files that depend on it - unless compiling that file succeeds. In general we would want #424, so that we can recompile dependencies even on an error, and show multiple errors per file.

Dealing with errors in the data model might be a pain, since there might not be valid core code when there is an error, unless we create a core stub that prints an error to stdout and exits. We could make it a Maybe type, but then that makes the rest of the compiler difficult. We could also just create a wrapper ModuleSuccess(mod: Module)/ModuleError(prior: Maybe<Module>, errors: [...]) so we could handle errors while still having the Module interface represent successful stages, and still allow other modules to depend on the last successful compilation of a module.

[Tim Whiting]

Some additional ideas when talking to Daan. Loaded lookup maps (Gamma/Newtypes/Synonyms) should really just be on-demand lookups on the [Module] that are publicly imported to the current module.