Improve performance of graph operations

[Radek Strnad]

Due to underlying data structures (Map) it takes O(n) to lookup a node in the graph. fp-ts iterates the map one by one and uses Eq to match the node. This is extremely limiting when having large graphs (thousands+ of nodes) and performing complex traverse operations.

Suggested improvements:

1. Provide io-ts codec which encodes / decodes arbitrary data type to string
2. Replace all occurances of Map with Record and use encoded strings as keys (lookup complexity between O(1) and O(log(n)))

import { Codec } from 'io-ts/Codec';

export interface Graph<Id, Edge, Node> {
  readonly _brand: unique symbol;
  readonly nodes: Record<string, NodeContext<Node>>;
  readonly edges: Record<string, Edge>;
}

type NodeContext<Node> = {
  data: Node;
  outgoing: Set<string>;
  incoming: Set<string>;
};

Usually only first function argument changes from E: Eq<Id> to C: Codec<string, string, Id>. The remaining arguments should remain the same same . The only problem I see is with potential collision when building what's been Direction<T> = { from: T, to: T } and now is string out of two strings (using derived Codec<string, string, string>. We can use some non-standard sequence of UTF characters which is highly unlikely to collide and write an explicit warning in the documentation.

Let me know your thoughts.

[Michael Bock]

Ok, this is an extremely good point. To be honest, I was not aware that fp-ts' lookup is O(n). But it totally makes sense because of the Eq.

I definitely like the proposed idea with the codecs. I'm not happy though with the issue that comes up for Direction, as you describe. In any way I plan to introduce Edge Id's. They're needed because graphs should be able to have multiple edges between the same nodes.
The clean way to solve this for now would be to introduce a new type: SpecialString with a smart constructor that check if the string does not contain that one magic delimiter character. But well, that would be not a nice API.

Another option would be to use a data structure based on "balanced 2-3 trees" (as e.g. PureScript's Map). But I would not know if that exists for ts/fp-ts.

So that's my spontaneous thought on that. As this would be a larger change. I'd propose: Let's wait a bit to investigate further on it.

But it's definitively a high prio thing, as I said. I wrote it with a wrong assumption.

[Radek Strnad]

I thought of similar approach as SpecialString, but I'm not exactly thrilled about the constructor processing / error handling penalty. I have a working version of the library already done & tested at https://github.com/stockstory/fp-ts-graph. Now I'm working on changes on our internal code to see performance impact. I'll report my findings soon.

[Michael Bock]

btw, why would we need a codec. Wouldn't encoding be enough? Thus maybe only Show?
I'll check out your fork...

[Michael Bock]

ok, I see. for e.g. reading the entries

[Radek Strnad]

Yeah, most destructor functions require Decoder.

[Michael Bock]

What do generally think about the idea that edges can have id's?

[Michael Bock]

I was too fast yesterday. Thought about it again. Even if edge ids are introduced, there's still the need for an additional record that has a connection (from, to) as key.

So we cannot get around the delimiter problem. And, yeah. I'm still not happy with the "rare unicode character hack". So I'm in favor of introducing a GraphId type with a smart constructor. It should be placed inside a separate module.

If the user is really sure that the delimiter does not appear in the string, something like "unsafeFromString" can be used to create a GraphId. But then the peril is on the user's side and the library code is clean.

[Michael Bock]

And now I thought about it once again, and see:

We can solve the lookup problem by using a nested Record data structure:

...
edges: Record<string, Record<string, Edge>>
...

whereas the outer string represents the "from" node id and the inner the "to" node id.

[Radek Strnad]

I like the nested Record approach. I'll update my code.

[Radek Strnad]

So I ran some benchmarks on orignal code and proposed code and found some interesting results (executed on i7-8750H).

                        0.2.2 (Map)       0.3.0-dev (Record)
Insert 10000 nodes        6517 ms            21273 ms
Insert 10000 edges       39047 ms           107744 ms
Lookup 10000 nodes        1985 ms               12 ms
Lookup 10000 edges        4848 ms               13 ms

For some reason building a Record is ~ 3 times as slow as building a Map. I'll have a look into it deeper, however if we should move forward with this change, we should introduce bulk insert functionality with significantly better performance.

[Michael Bock]

Oh, surprising. But still, I prefer to have constant performance decrease in insertion more than having a linear lookup behavior. And true, maybe bulk insertion may improve that.

Now I worked on the layout for "v1". It's a draft for now, only types, no implementations. But that's more or less how I imagine the next version of the library. https://github.com/no-day/fp-ts-graph/blob/v1

Main features included:

• Support for multiedge, non-directed, acyclic graph qualities. Graph config is kept at the typelevel (https://github.com/no-day/fp-ts-graph/blob/v1/src/Graph/type.ts#L16)
• Bulk insertion ("fromEntries", check out the new readme or example at https://github.com/no-day/fp-ts-graph/blob/v1/examples/build-graph.ts
• Consistent naming of CRUD operations (as described here: #13)
• More descriptive Error handling (Either instead of Option)
• Fast lookups (as discussed here)
• Better code/module organisation

[Radek Strnad]

I spent more time profiling both versions of the code. Insert of Map version is in fact O(n^2) since fp-ts performs sequential scan with Eqs for every inserted item. However it still outperforms native object (Record) creation when building the data structure. I tried to rewrite some of fp-ts's Record functions both with Object.assign and with {...spread} operator with no measurable speed improvement.

Map seems to be more performant than Record when used directly (and some blog posts confirm my findings). However I also think the io-ts encoding / decoding to string is good idea, because that way we can avoid fp-ts's sequential scans by using native JavaScript Map<string, ???> access methods and get significantly better results.

Another option is to use 3rd party library like immutable.js, which claims O(log32 n) for get / set using hash-array mapped trie. The library further implements bulk operations, which would be perfect for fromEntries ticket.

[Michael Bock]

I did not go into the details here. I just think that performance of immutable operations is not a priority in fp-ts (yet). Map has the Eq overhead. Record is tied to string, so the overhead is only in the recreation of the record on every change. One thing I notices on the last profiling journey I made: It really makes a difference which engine you use. I remember something like that Chrome does a lot of optimization under the hood.

You mention immutable.js, another thing I had in mind for a while is to use immer.js to improve fp-ts immutablity operations. If you're into this currently, maybe it's worth to give it a try, It does some pretty neat proxy/copy-on-write magic to turn mutable code into fast immutable code.

My focus of the library right now is not performance but rather consistency of the API and type safety. But of course I'm open to things that make it faster :)

We can aim to release your proposal as "0.3.0", as the "1.0.0" that I started to work on will take a while.

[Radek Strnad]

I tried several approaches to fight the suboptimal performance. Without having to rewrite all functions I picked insert node function with encoding key from arbitrary object to string using io-ts, which is basically insert in a map at the moment (version 0.2.2). For full transparency I was using { first: number; second: number; } as a composite key. My baseline was standard mutable Map which performs pretty well.

I then tried same appropach as fp-ts uses to achieve Map immmutability - create a shallow copy of the map each time you mutate it (new Map(oldMap)). I also tried Immer as you suggested, however I got worse results than with native Map made immutable. Furthermore the library doesn't guarantee immutability at compile time by type design, but checks access using immutable mutator functions at runtime and can throw an error, which goes against FP principles. Also it requires executing magic command at the start just to start working for Maps, otherwise it again throws runtime error.

I got pretty amazing results with immutable.js library which I will use for my PR. The code is now roughly 2-3 orders of magnitude faster than before for 10k records and then it scales almost linearly:

        Mutable map     Immutable map       Immer       Immutable.js
1k             3 ms             41 ms       91 ms              11 ms
10k            9 ms           4899 ms     9833 ms              50 ms
100k          58 ms         805635 ms  1188372 ms             168 ms
1m           655 ms            N/A         N/A               1906 ms

Just to recap a benchmark after change for Immutable.js:

Immutable.js + io.ts
        insert nodes    insert edges    lookup nodes    lookup edges
1k      13              27              1               2
10k     59              85              8               14
100k    231             653             75              98
1m      2637            9673            1180            3420