Monorepo of isomorphic utility functions
This library is meant to replace all global JavaScript functions with isomorphic module imports. Additionally, it implements several performance-oriented utility modules. Most noteworthy are the binary encoding/decoding modules [lib0/encoding] / [lib0/decoding], the randomized testing framework [lib0/testing], the fast Pseudo Random Number Generator [lib0/PRNG], the small socket.io alternative [lib0/websocket], and the logging module [lib0/logging] that allows colorized logging in all environments. Lib0 has only one dependency, which is also from the author of lib0. If lib0 is transpiled with rollup or webpack, very little code is produced because of the way that it is written. All exports are pure and are removed by transpilers that support dead code elimination. Here is an example of how dead code elemination and mangling optimizes code from lib0:
// How the code is optimized by transpilers:
// lib0/json.js
export const stringify = JSON.stringify
export const parse = JSON.parse
// index.js
import * as json from 'lib0/json.js'
export const f = (arg1, arg2) => json.stringify(arg1) + json.stringify(arg2)
// compiled with rollup and uglifyjs:
const s=JSON.stringify,f=(a,b)=>s(a)+s(b)
export {f}
Each function in this library is tested thoroughly and is not deoptimized by v8 (except some logging and comparison functions that can't be implemented without deoptimizations). This library implements its own test suite that is designed for randomized testing and inspecting performance issues.
node --trace-deop
andnode --trace-opt
- https://youtu.be/IFWulQnM5E0 Good intro video
- https://github.com/thlorenz/v8-perf
- https://github.com/thlorenz/deoptigate - A great tool for investigating deoptimizations
- https://github.com/vhf/v8-bailout-reasons - Description of some deopt messages
The code style might be a bit different from what you are used to. Stay open. Most of the design choices have been thought through. The purpose of this code style is to create code that is optimized by the compiler and that results in small code bundles when used with common module bundlers. Keep that in mind when reading the library.
- Modules should only export pure functions and constants. This way the module bundler can eliminate dead code. The statement
const x = someCondition ? A : B
cannot be eleminated, because it is tied to a condition. - Use Classes for structuring data. Classes are well supported by jsdoc and are immediately optimized by the compiler. I.e. prefer
class Coord { constructor (x, y) { this.x = x; this.y = y} }
instead of{ x: x, y: y }
, because the compiler needs to be assured that the order of properties does not change.{ y: y, x: x }
has a different hidden class than{ x: x, y: y }
, which will lead to code deoptimizations if their use is alternated. - The user of your module should never create data objects with the
new
keyword. Prefer exporting factory functions likeconst createCoordinate = (x, y) => new Coord(x, y)
. - The use of class methods is discouraged, because method names can't be mangled or removed by dead code elimination.
- The only acceptable use of methods is when two objects implement functionality differently.
E.g.
class Duck { eat () { swallow() } }
andclass Cow { eat () { chew() } }
have the same signature, but implement it differently. - Prefer
const
variable declarations. Uselet
only in loops.const
always leads to easier code. - Keep the potential execution stack small and compartmentalized. Nobody wants to debug spaghetti code.
- Give proper names to your functions and ask yourself if you would know what the function does if you saw it in the execution stack.
- Avoid recursion. There is a stack limit in most browsers and not every recursive function is optimized by the compiler.
- Semicolons are superfluous. Lint with https://standardjs.com/
lib0
contains isomorphic modules that work in nodejs, the browser, and other environments. It exports modules as the commonjs
and the new esm module
format.
If possible,
ESM module
import module from 'lib0/[module]' // automatically resolves to lib0/[module].js
CommonJS
require('lib0/[module]') // automatically resolves to lib0/dist/[module].cjs
Manual
Automatically resolving to commonjs
and esm modules
is implemented using conditional exports which is available in node>=v12
. If support for older versions is required, then it is recommended to define the location of the module manually:
import module from 'lib0/[module].js'
// require('lib0/dist/[module].cjs')
[lib0/array] Utility module to work with Arrays.
import * as array from 'lib0/array.js'
Return the last element of an array. The element must exist
Append elements from src to dest
Transforms something array-like to an actual Array.
True iff condition holds on every element in the Array.
True iff condition holds on some element in the Array.
array.last(arr: Array<L>): L
array.create(): Array<C>
array.copy(a: Array<D>): Array<D>
array.appendTo(dest: Array<M>, src: Array<M>)
array.from(arraylike: ArrayLike<T>|Iterable<T>): T
array.every(arr: Array<ITEM>, f: function(ITEM, number, Array<ITEM>):boolean): boolean
array.some(arr: Array<S>, f: function(S, number, Array<S>):boolean): boolean
array.equalFlat(a: Array<ELEM>, b: Array<ELEM>): boolean
array.flatten(arr: Array<Array<ELEM>>): Array<ELEM>
[lib0/binary] Binary data constants.
import * as binary from 'lib0/binary.js'
n-th bit activated.
First n bits activated.
binary.BIT1: number
binary.BIT2
binary.BIT3
binary.BIT4
binary.BIT5
binary.BIT6
binary.BIT7
binary.BIT8
binary.BIT9
binary.BIT10
binary.BIT11
binary.BIT12
binary.BIT13
binary.BIT14
binary.BIT15
binary.BIT16
binary.BIT17
binary.BIT18
binary.BIT19
binary.BIT20
binary.BIT21
binary.BIT22
binary.BIT23
binary.BIT24
binary.BIT25
binary.BIT26
binary.BIT27
binary.BIT28
binary.BIT29
binary.BIT30
binary.BIT31
binary.BIT32
binary.BITS0: number
binary.BITS1
binary.BITS2
binary.BITS3
binary.BITS4
binary.BITS5
binary.BITS6
binary.BITS7
binary.BITS8
binary.BITS9
binary.BITS10
binary.BITS11
binary.BITS12
binary.BITS13
binary.BITS14
binary.BITS15
binary.BITS16
binary.BITS17
binary.BITS18
binary.BITS19
binary.BITS20
binary.BITS21
binary.BITS22
binary.BITS23
binary.BITS24
binary.BITS25
binary.BITS26
binary.BITS27
binary.BITS28
binary.BITS29
binary.BITS30
binary.BITS31: number
binary.BITS32: number
[lib0/broadcastchannel] Helpers for cross-tab communication using broadcastchannel with LocalStorage fallback.
import * as broadcastchannel from 'lib0/broadcastchannel.js'
// In browser window A:
broadcastchannel.subscribe('my events', data => console.log(data))
broadcastchannel.publish('my events', 'Hello world!') // => A: 'Hello world!' fires synchronously in same tab
// In browser window B:
broadcastchannel.publish('my events', 'hello from tab B') // => A: 'hello from tab B'
Subscribe to global
publish
events.Unsubscribe from
publish
global events.Publish data to all subscribers (including subscribers on this tab)
broadcastchannel.subscribe(room: string, f: function(any):any)
broadcastchannel.unsubscribe(room: string, f: function(any):any)
broadcastchannel.publish(room: string, data: any)
[lib0/buffer] Utility functions to work with buffers (Uint8Array).
import * as buffer from 'lib0/buffer.js'
Create Uint8Array with initial content from buffer
Create Uint8Array with initial content from buffer
Copy the content of an Uint8Array view to a new ArrayBuffer.
Encode anything as a UInt8Array. It's a pun on typescripts's
any
type. See encoding.writeAny for more information.Decode an any-encoded value.
buffer.createUint8ArrayFromLen(len: number)
buffer.createUint8ArrayViewFromArrayBuffer(buffer: ArrayBuffer, byteOffset: number, length: number)
buffer.createUint8ArrayFromArrayBuffer(buffer: ArrayBuffer)
buffer.toBase64
buffer.fromBase64
buffer.copyUint8Array(uint8Array: Uint8Array): Uint8Array
buffer.encodeAny(data: any): Uint8Array
buffer.decodeAny(buf: Uint8Array): any
[lib0/component] Web components.
import * as component from 'lib0/component.js'
component.registry
component.define(name: string, constr: any, opts: ElementDefinitionOptions)
component.whenDefined(name: string): Promise<void>
new component.Lib0Component(state: S)
component.Lib0Component#state: S|null
component.Lib0Component#setState(state: S, forceStateUpdate: boolean)
component.Lib0Component#updateState(stateUpdate: any)
component.createComponent(name: string, cnf: module:component~CONF<T>): Class<module:component.Lib0Component>
component.createComponentDefiner(definer: function)
component.defineListComponent
component.defineLazyLoadingComponent
[lib0/conditions] Often used conditions.
import * as conditions from 'lib0/conditions.js'
conditions.undefinedToNull
[lib0/decoding] Efficient schema-less binary decoding with support for variable length encoding.
import * as decoding from 'lib0/decoding.js'
Use [lib0/decoding] with [lib0/encoding]. Every encoding function has a corresponding decoding function.
Encodes numbers in little-endian order (least to most significant byte order) and is compatible with Golang's binary encoding (https://golang.org/pkg/encoding/binary/) which is also used in Protocol Buffers.
// encoding step
const encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, 256)
encoding.writeVarString(encoder, 'Hello world!')
const buf = encoding.toUint8Array(encoder)
// decoding step
const decoder = new decoding.createDecoder(buf)
decoding.readVarUint(decoder) // => 256
decoding.readVarString(decoder) // => 'Hello world!'
decoding.hasContent(decoder) // => false - all data is read
A Decoder handles the decoding of an Uint8Array.
Decoding target.
Current decoding position.
Clone a decoder instance. Optionally set a new position parameter.
Create an Uint8Array view of the next
len
bytes and advance the position bylen
.Important: The Uint8Array still points to the underlying ArrayBuffer. Make sure to discard the result as soon as possible to prevent any memory leaks. Use
buffer.copyUint8Array
to copy the result into a new Uint8Array.Read variable length Uint8Array.
Important: The Uint8Array still points to the underlying ArrayBuffer. Make sure to discard the result as soon as possible to prevent any memory leaks. Use
buffer.copyUint8Array
to copy the result into a new Uint8Array.Read the rest of the content as an ArrayBuffer
Skip one byte, jump to the next position.
Read one byte as unsigned integer.
Read 2 bytes as unsigned integer.
Read 4 bytes as unsigned integer.
Read 4 bytes as unsigned integer in big endian order. (most significant byte first)
Look ahead without incrementing the position to the next byte and read it as unsigned integer.
Look ahead without incrementing the position to the next byte and read it as unsigned integer.
Look ahead without incrementing the position to the next byte and read it as unsigned integer.
Read unsigned integer (32bit) with variable length. 1/8th of the storage is used as encoding overhead.
- numbers < 2^7 is stored in one bytlength
- numbers < 2^14 is stored in two bylength
Read signed integer (32bit) with variable length. 1/8th of the storage is used as encoding overhead.
- numbers < 2^7 is stored in one bytlength
- numbers < 2^14 is stored in two bylength
Look ahead and read varUint without incrementing position
Look ahead and read varUint without incrementing position
Read string of variable length
- varUint is used to store the length of the string
Transforming utf8 to a string is pretty expensive. The code performs 10x better when String.fromCodePoint is fed with all characters as arguments. But most environments have a maximum number of arguments per functions. For effiency reasons we apply a maximum of 10000 characters at once.
Look ahead and read varString without incrementing position
T must not be null.
Current state
Current state
Current state
Decoding target.
Current decoding position.
Decoding target.
Current decoding position.
Decoding target.
Current decoding position.
Decoding target.
Current decoding position.
Decoding target.
Current decoding position.
Decoding target.
Current decoding position.
new decoding.Decoder(uint8Array: Uint8Array)
decoding.Decoder#arr: Uint8Array
decoding.Decoder#pos: number
decoding.createDecoder(uint8Array: Uint8Array): module:decoding.Decoder
decoding.hasContent(decoder: module:decoding.Decoder): boolean
decoding.clone(decoder: module:decoding.Decoder, newPos: number): module:decoding.Decoder
decoding.readUint8Array(decoder: module:decoding.Decoder, len: number): Uint8Array
decoding.readVarUint8Array(decoder: module:decoding.Decoder): Uint8Array
decoding.readTailAsUint8Array(decoder: module:decoding.Decoder): Uint8Array
decoding.skip8(decoder: module:decoding.Decoder): number
decoding.readUint8(decoder: module:decoding.Decoder): number
decoding.readUint16(decoder: module:decoding.Decoder): number
decoding.readUint32(decoder: module:decoding.Decoder): number
decoding.readUint32BigEndian(decoder: module:decoding.Decoder): number
decoding.peekUint8(decoder: module:decoding.Decoder): number
decoding.peekUint16(decoder: module:decoding.Decoder): number
decoding.peekUint32(decoder: module:decoding.Decoder): number
decoding.readVarUint(decoder: module:decoding.Decoder): number
decoding.readVarInt(decoder: module:decoding.Decoder): number
decoding.peekVarUint(decoder: module:decoding.Decoder): number
decoding.peekVarInt(decoder: module:decoding.Decoder): number
decoding.readVarString(decoder: module:decoding.Decoder): String
decoding.peekVarString(decoder: module:decoding.Decoder): string
decoding.readFromDataView(decoder: module:decoding.Decoder, len: number): DataView
decoding.readFloat32(decoder: module:decoding.Decoder)
decoding.readFloat64(decoder: module:decoding.Decoder)
decoding.readBigInt64(decoder: module:decoding.Decoder)
decoding.readBigUint64(decoder: module:decoding.Decoder)
decoding.readAny(decoder: module:decoding.Decoder)
new decoding.RleDecoder(uint8Array: Uint8Array, reader: function(module:decoding.Decoder):T)
decoding.RleDecoder#s: T|null
decoding.RleDecoder#read()
decoding.RleDecoder#s: T
new decoding.IntDiffDecoder(uint8Array: Uint8Array, start: number)
decoding.IntDiffDecoder#s: number
decoding.IntDiffDecoder#read(): number
new decoding.RleIntDiffDecoder(uint8Array: Uint8Array, start: number)
decoding.RleIntDiffDecoder#s: number
decoding.RleIntDiffDecoder#read(): number
decoding.RleIntDiffDecoder#s: number
new decoding.UintOptRleDecoder(uint8Array: Uint8Array)
decoding.UintOptRleDecoder#s: number
decoding.UintOptRleDecoder#read()
decoding.UintOptRleDecoder#s: number
new decoding.IncUintOptRleDecoder(uint8Array: Uint8Array)
decoding.IncUintOptRleDecoder#s: number
decoding.IncUintOptRleDecoder#read()
new decoding.IntDiffOptRleDecoder(uint8Array: Uint8Array)
decoding.IntDiffOptRleDecoder#s: number
decoding.IntDiffOptRleDecoder#read(): number
new decoding.StringDecoder(uint8Array: Uint8Array)
decoding.StringDecoder#spos: number
decoding.StringDecoder#read(): string
decoding.RleDecoder#arr: Uint8Array
decoding.RleDecoder#pos: number
decoding.IntDiffDecoder#arr: Uint8Array
decoding.IntDiffDecoder#pos: number
decoding.RleIntDiffDecoder#arr: Uint8Array
decoding.RleIntDiffDecoder#pos: number
decoding.UintOptRleDecoder#arr: Uint8Array
decoding.UintOptRleDecoder#pos: number
decoding.IncUintOptRleDecoder#arr: Uint8Array
decoding.IncUintOptRleDecoder#pos: number
decoding.IntDiffOptRleDecoder#arr: Uint8Array
decoding.IntDiffOptRleDecoder#pos: number
[lib0/diff] Efficient diffs.
import * as diff from 'lib0/diff.js'
Create a diff between two strings. This diff implementation is highly efficient, but not very sophisticated.
Create a diff between two arrays. This diff implementation is highly efficient, but not very sophisticated.
Note: This is basically the same function as above. Another function was created so that the runtime can better optimize these function calls.
diff.simpleDiffString(a: string, b: string): module:diff~SimpleDiff<string>
diff.simpleDiff
diff.simpleDiffArray(a: Array<T>, b: Array<T>, compare: function(T, T):boolean): module:diff~SimpleDiff<Array<T>>
[lib0/dom] Utility module to work with the DOM.
import * as dom from 'lib0/dom.js'
dom.doc: Document
dom.createElement
dom.createDocumentFragment
dom.createTextNode
dom.domParser
dom.emitCustomEvent
dom.setAttributes
dom.setAttributesMap
dom.fragment
dom.append
dom.remove
dom.addEventListener
dom.removeEventListener
dom.addEventListeners
dom.removeEventListeners
dom.element
dom.canvas
dom.text
dom.pairToStyleString
dom.pairsToStyleString
dom.mapToStyleString
dom.querySelector
dom.querySelectorAll
dom.getElementById
dom.parseFragment
dom.childNodes: any
dom.parseElement
dom.replaceWith
dom.insertBefore
dom.appendChild
dom.ELEMENT_NODE
dom.TEXT_NODE
dom.CDATA_SECTION_NODE
dom.COMMENT_NODE
dom.DOCUMENT_NODE
dom.DOCUMENT_TYPE_NODE
dom.DOCUMENT_FRAGMENT_NODE
dom.checkNodeType(node: any, type: number)
dom.isParentOf(parent: Node, child: HTMLElement)
[lib0/encoding] Efficient schema-less binary encoding with support for variable length encoding.
import * as encoding from 'lib0/encoding.js'
Use [lib0/encoding] with [lib0/decoding]. Every encoding function has a corresponding decoding function.
Encodes numbers in little-endian order (least to most significant byte order) and is compatible with Golang's binary encoding (https://golang.org/pkg/encoding/binary/) which is also used in Protocol Buffers.
// encoding step
const encoder = new encoding.createEncoder()
encoding.writeVarUint(encoder, 256)
encoding.writeVarString(encoder, 'Hello world!')
const buf = encoding.toUint8Array(encoder)
// decoding step
const decoder = new decoding.createDecoder(buf)
decoding.readVarUint(decoder) // => 256
decoding.readVarString(decoder) // => 'Hello world!'
decoding.hasContent(decoder) // => false - all data is read
A BinaryEncoder handles the encoding to an Uint8Array.
The current length of the encoded data.
Transform to Uint8Array.
Write one byte to the encoder.
Write one byte at a specific position. Position must already be written (i.e. encoder.length > pos)
Write one byte as an unsigned integer.
Write one byte as an unsigned Integer at a specific location.
Write two bytes as an unsigned integer.
Write two bytes as an unsigned integer at a specific location.
Write two bytes as an unsigned integer
Write two bytes as an unsigned integer in big endian order. (most significant byte first)
Write two bytes as an unsigned integer at a specific location.
Write a variable length unsigned integer.
Encodes integers in the range from [0, 4294967295] / [0, 0xffffffff]. (max 32 bit unsigned integer)
Write a variable length integer.
Encodes integers in the range from [-2147483648, -2147483647].
We don't use zig-zag encoding because we want to keep the option open to use the same function for BigInt and 53bit integers (doubles).
We use the 7th bit instead for signaling that this is a negative number.
Write a variable length string.
Write the content of another Encoder.
Append fixed-length Uint8Array to the encoder.
Append an Uint8Array to Encoder.
Create an DataView of the next
len
bytes. Use it to write data after calling this function.// write float32 using DataView const dv = writeOnDataView(encoder, 4) dv.setFloat32(0, 1.1) // read float32 using DataView const dv = readFromDataView(encoder, 4) dv.getFloat32(0) // => 1.100000023841858 (leaving it to the reader to find out why this is the correct result)
Encode data with efficient binary format.
Differences to JSON: • Transforms data to a binary format (not to a string) • Encodes undefined, NaN, and ArrayBuffer (these can't be represented in JSON) • Numbers are efficiently encoded either as a variable length integer, as a 32 bit float, as a 64 bit float, or as a 64 bit bigint.
Encoding table:
Data Type Prefix Encoding Method Comment undefined 127 Functions, symbol, and everything that cannot be identified is encoded as undefined null 126 integer 125 writeVarInt Only encodes 32 bit signed integers float32 124 writeFloat32 float64 123 writeFloat64 bigint 122 writeBigInt64 boolean (false) 121 True and false are different data types so we save the following byte boolean (true) 120 - 0b01111000 so the last bit determines whether true or false string 119 writeVarString object<string,any> 118 custom Writes {length} then {length} key-value pairs array 117 custom Writes {length} then {length} json values Uint8Array 116 writeVarUint8Array We use Uint8Array for any kind of binary data Reasons for the decreasing prefix: We need the first bit for extendability (later we may want to encode the prefix with writeVarUint). The remaining 7 bits are divided as follows: [0-30] the beginning of the data range is used for custom purposes (defined by the function that uses this library) [31-127] the end of the data range is used for data encoding by lib0/encoding.js
Now come a few stateful encoder that have their own classes.
Current state
Basic diff decoder using variable length encoding.
Encodes the values [3, 1100, 1101, 1050, 0] to [3, 1097, 1, -51, -1050] using writeVarInt.
Current state
A combination of IntDiffEncoder and RleEncoder.
Basically first writes the IntDiffEncoder and then counts duplicate diffs using RleEncoding.
Encodes the values [1,1,1,2,3,4,5,6] as [1,1,0,2,1,5] (RLE([1,0,0,1,1,1,1,1]) ⇒ RleIntDiff[1,1,0,2,1,5])
Current state
Optimized Rle encoder that does not suffer from the mentioned problem of the basic Rle encoder.
Internally uses VarInt encoder to write unsigned integers. If the input occurs multiple times, we write write it as a negative number. The UintOptRleDecoder then understands that it needs to read a count.
Encodes [1,2,3,3,3] as [1,2,-3,3] (once 1, once 2, three times 3)
Increasing Uint Optimized RLE Encoder
The RLE encoder counts the number of same occurences of the same value. The IncUintOptRle encoder counts if the value increases. I.e. 7, 8, 9, 10 will be encoded as [-7, 4]. 1, 3, 5 will be encoded as [1, 3, 5].
A combination of the IntDiffEncoder and the UintOptRleEncoder.
The count approach is similar to the UintDiffOptRleEncoder, but instead of using the negative bitflag, it encodes in the LSB whether a count is to be read. Therefore this Encoder only supports 31 bit integers!
Encodes [1, 2, 3, 2] as [3, 1, 6, -1] (more specifically [(1 << 1) | 1, (3 << 0) | 0, -1])
Internally uses variable length encoding. Contrary to normal UintVar encoding, the first byte contains:
- 1 bit that denotes whether the next value is a count (LSB)
- 1 bit that denotes whether this value is negative (MSB - 1)
- 1 bit that denotes whether to continue reading the variable length integer (MSB)
Therefore, only five bits remain to encode diff ranges.
Use this Encoder only when appropriate. In most cases, this is probably a bad idea.
Optimized String Encoder.
Encoding many small strings in a simple Encoder is not very efficient. The function call to decode a string takes some time and creates references that must be eventually deleted. In practice, when decoding several million small strings, the GC will kick in more and more often to collect orphaned string objects (or maybe there is another reason?).
This string encoder solves the above problem. All strings are concatenated and written as a single string using a single encoding call.
The lengths are encoded using a UintOptRleEncoder.
new encoding.Encoder()
encoding.Encoder#bufs: Array<Uint8Array>
encoding.createEncoder(): module:encoding.Encoder
encoding.length(encoder: module:encoding.Encoder): number
encoding.toUint8Array(encoder: module:encoding.Encoder): Uint8Array
encoding.write(encoder: module:encoding.Encoder, num: number)
encoding.set(encoder: module:encoding.Encoder, pos: number, num: number)
encoding.writeUint8(encoder: module:encoding.Encoder, num: number)
encoding.setUint8(encoder: module:encoding.Encoder, pos: number, num: number)
encoding.writeUint16(encoder: module:encoding.Encoder, num: number)
encoding.setUint16(encoder: module:encoding.Encoder, pos: number, num: number)
encoding.writeUint32(encoder: module:encoding.Encoder, num: number)
encoding.writeUint32BigEndian(encoder: module:encoding.Encoder, num: number)
encoding.setUint32(encoder: module:encoding.Encoder, pos: number, num: number)
encoding.writeVarUint(encoder: module:encoding.Encoder, num: number)
encoding.writeVarInt(encoder: module:encoding.Encoder, num: number)
encoding.writeVarString(encoder: module:encoding.Encoder, str: String)
encoding.writeBinaryEncoder(encoder: module:encoding.Encoder, append: module:encoding.Encoder)
encoding.writeUint8Array(encoder: module:encoding.Encoder, uint8Array: Uint8Array)
encoding.writeVarUint8Array(encoder: module:encoding.Encoder, uint8Array: Uint8Array)
encoding.writeOnDataView(encoder: module:encoding.Encoder, len: number): DataView
encoding.writeFloat32(encoder: module:encoding.Encoder, num: number)
encoding.writeFloat64(encoder: module:encoding.Encoder, num: number)
encoding.writeBigInt64(encoder: module:encoding.Encoder, num: bigint)
encoding.writeBigUint64(encoder: module:encoding.Encoder, num: bigint)
encoding.writeAny(encoder: module:encoding.Encoder, data: undefined|null|number|bigint|boolean|string|Object<string,any>|Array<any>|Uint8Array)
new encoding.RleEncoder(writer: function(module:encoding.Encoder, T):void)
encoding.RleEncoder#s: T|null
encoding.RleEncoder#write(v: T)
new encoding.IntDiffEncoder(start: number)
encoding.IntDiffEncoder#s: number
encoding.IntDiffEncoder#write(v: number)
new encoding.RleIntDiffEncoder(start: number)
encoding.RleIntDiffEncoder#s: number
encoding.RleIntDiffEncoder#write(v: number)
new encoding.UintOptRleEncoder()
encoding.UintOptRleEncoder#s: number
encoding.UintOptRleEncoder#write(v: number)
encoding.UintOptRleEncoder#toUint8Array()
new encoding.IncUintOptRleEncoder()
encoding.IncUintOptRleEncoder#s: number
encoding.IncUintOptRleEncoder#write(v: number)
encoding.IncUintOptRleEncoder#toUint8Array()
new encoding.IntDiffOptRleEncoder()
encoding.IntDiffOptRleEncoder#s: number
encoding.IntDiffOptRleEncoder#write(v: number)
encoding.IntDiffOptRleEncoder#toUint8Array()
new encoding.StringEncoder()
encoding.StringEncoder#sarr: Array<string>
encoding.StringEncoder#write(string: string)
encoding.StringEncoder#toUint8Array()
encoding.RleEncoder#bufs: Array<Uint8Array>
encoding.IntDiffEncoder#bufs: Array<Uint8Array>
encoding.RleIntDiffEncoder#bufs: Array<Uint8Array>
[lib0/map] Isomorphic module to work access the environment (query params, env variables).
import * as map from 'lib0/environment.js'
map.isNode
map.isBrowser
map.isMac
map.hasParam
map.getParam
map.getVariable
map.getConf(name: string): string|null
map.hasConf
map.production
[lib0/error] Error helpers.
import * as error from 'lib0/error.js'
error.create
error.methodUnimplemented
error.unexpectedCase
[lib0/eventloop] Utility module to work with EcmaScript's event loop.
import * as eventloop from 'lib0/eventloop.js'
Note: this is experimental and is probably only useful in browsers.
eventloop.enqueue(f: function():void)
eventloop#destroy()
eventloop.timeout(timeout: number, callback: function): module:eventloop~TimeoutObject
eventloop.interval(timeout: number, callback: function): module:eventloop~TimeoutObject
eventloop.Animation
eventloop.animationFrame(cb: function(number):void): module:eventloop~TimeoutObject
eventloop.idleCallback(cb: function): module:eventloop~TimeoutObject
eventloop.createDebouncer(timeout: number): function(function():void):void
[lib0/function] Common functions and function call helpers.
import * as function from 'lib0/function.js'
Calls all functions in
fs
with args. Only throws after all functions were called.
function.callAll(fs: Array<function>, args: Array<any>)
function.nop
function.apply(f: function():T): T
function.id(a: A): A
function.equalityStrict(a: T, b: T): boolean
function.equalityFlat(a: Array<T>|object, b: Array<T>|object): boolean
function.equalityDeep(a: any, b: any): boolean
[lib0/lib0] Experimental method to import lib0.
import * as lib0 from 'lib0/index.js'
Not recommended if the module bundler doesn't support dead code elimination.
[lib0/idb] Helpers to work with IndexedDB.
import * as idb from 'lib0/indexeddb.js'
idb.rtop
idb.openDB
idb.deleteDB
idb.createStores
idb.transact(db: IDBDatabase, stores: Array<string>, access: "readwrite"|"readonly"): Array<IDBObjectStore>
idb.count
idb.get
idb.del
idb.put
idb.add
idb.addAutoKey
idb.getAll
idb.getAllKeys
idb.queryFirst(store: IDBObjectStore, query: IDBKeyRange|null, direction: 'next'|'prev'|'nextunique'|'prevunique'): Promise<any>
idb.getLastKey(store: IDBObjectStore): Promise<any>
idb.getFirstKey(store: IDBObjectStore): Promise<any>
idb.getAllKeysValues
idb.iterate
idb.iterateKeys
idb.getStore
idb.createIDBKeyRangeBound
idb.createIDBKeyRangeUpperBound
idb.createIDBKeyRangeLowerBound
[lib0/isomorphic] Isomorphic library exports from isomorphic.js.
import * as isomorphic from 'lib0/isomorphic.js'
[lib0/iterator] Utility module to create and manipulate Iterators.
import * as iterator from 'lib0/iterator.js'
iterator.mapIterator(iterator: Iterator<T>, f: function(T):R): IterableIterator<R>
iterator.createIterator(next: function():IteratorResult<T>): IterableIterator<T>
iterator.iteratorFilter(iterator: Iterator<T>, filter: function(T):boolean)
iterator.iteratorMap(iterator: Iterator<T>, fmap: function(T):M)
[lib0/json] JSON utility functions.
import * as json from 'lib0/json.js'
Transform JavaScript object to JSON.
Parse JSON object.
json.stringify(object: any): string
json.parse(json: string): any
[lib0/logging] Isomorphic logging module with support for colors!
import * as logging from 'lib0/logging.js'
logging.BOLD
logging.UNBOLD
logging.BLUE
logging.GREY
logging.GREEN
logging.RED
logging.PURPLE
logging.ORANGE
logging.UNCOLOR
logging.print(args: Array<string|Symbol|Object|number>)
logging.warn(args: Array<string|Symbol|Object|number>)
logging.printError(err: Error)
logging.printImg(url: string, height: number)
logging.printImgBase64(base64: string, height: number)
logging.group(args: Array<string|Symbol|Object|number>)
logging.groupCollapsed(args: Array<string|Symbol|Object|number>)
logging.groupEnd
logging.printDom(createNode: function():Node)
logging.printCanvas(canvas: HTMLCanvasElement, height: number)
logging.vconsoles
new logging.VConsole(dom: Element)
logging.VConsole#ccontainer: Element
logging.VConsole#group(args: Array<string|Symbol|Object|number>, collapsed: boolean)
logging.VConsole#groupCollapsed(args: Array<string|Symbol|Object|number>)
logging.VConsole#groupEnd()
logging.VConsole#print(args: Array<string|Symbol|Object|number>)
logging.VConsole#printError(err: Error)
logging.VConsole#printImg(url: string, height: number)
logging.VConsole#printDom(node: Node)
logging.VConsole#destroy()
logging.createVConsole(dom: Element)
logging.createModuleLogger(moduleName: string): function(...any):void
[lib0/map] Utility module to work with key-value stores.
import * as map from 'lib0/map.js'
Creates a new Map instance.
Copy a Map object into a fresh Map object.
Get map property. Create T if property is undefined and set T on map.
const listeners = map.setIfUndefined(events, 'eventName', set.create) listeners.add(listener)
Creates an Array and populates it with the content of all key-value pairs using the
f(value, key)
function.Tests whether any key-value pairs pass the test implemented by
f(value, key)
.Tests whether all key-value pairs pass the test implemented by
f(value, key)
.
map.create(): Map<any, any>
map.copy(m: Map<X,Y>): Map<X,Y>
map.setIfUndefined(map: Map<K, T>, key: K, createT: function():T): T
map.map(m: Map<K,V>, f: function(V,K):R): Array<R>
map.any(m: Map<K,V>, f: function(V,K):boolean): boolean
map.all(m: Map<K,V>, f: function(V,K):boolean): boolean
[lib0/math] Common Math expressions.
import * as math from 'lib0/math.js'
Base 10 exponential function. Returns the value of 10 raised to the power of pow.
math.floor
math.ceil
math.abs
math.imul
math.round
math.log10
math.log2
math.log
math.sqrt
math.add(a: number, b: number): number
math.min(a: number, b: number): number
math.max(a: number, b: number): number
math.isNaN
math.pow
math.exp10(exp: number): number
math.sign
math.isNegativeZero(n: number): boolean
[lib0/metric] Utility module to convert metric values.
import * as metric from 'lib0/metric.js'
Calculate the metric prefix for a number. Assumes E.g.
prefix(1000) = { n: 1, prefix: 'k' }
metric.yotta
metric.zetta
metric.exa
metric.peta
metric.tera
metric.giga
metric.mega
metric.kilo
metric.hecto
metric.deca
metric.deci
metric.centi
metric.milli
metric.micro
metric.nano
metric.pico
metric.femto
metric.atto
metric.zepto
metric.yocto
metric.prefix(n: number, baseMultiplier: number): {n:number,prefix:string}
[lib0/mutex] Mutual exclude for JavaScript.
import * as mutex from 'lib0/mutex.js'
Creates a mutual exclude function with the following property:
const mutex = createMutex() mutex(() => { // This function is immediately executed mutex(() => { // This function is not executed, as the mutex is already active. }) })
mutex.createMutex(): mutex
[lib0/number]
import * as number from 'lib0/number.js'
number.MAX_SAFE_INTEGER
number.MIN_SAFE_INTEGER
number.LOWEST_INT32
number.HIGHEST_INT32: number
number.isInteger
number.isNaN
[lib0/object] Utility functions for working with EcmaScript objects.
import * as object from 'lib0/object.js'
Object.assign
Calls
Object.prototype.hasOwnProperty
.
object.create(): Object<string,any>
object.assign
object.keys(obj: Object<string,any>)
object.forEach(obj: Object<string,any>, f: function(any,string):any)
object.map(obj: Object<string,any>, f: function(any,string):R): Array<R>
object.length(obj: Object<string,any>): number
object.some(obj: Object<string,any>, f: function(any,string):boolean): boolean
object.every(obj: Object<string,any>, f: function(any,string):boolean): boolean
object.hasProperty(obj: any, key: string|symbol): boolean
object.equalFlat(a: Object<string,any>, b: Object<string,any>): boolean
[lib0/observable] Observable class prototype.
import * as observable from 'lib0/observable.js'
Handles named events.
Emit a named event. All registered event listeners that listen to the specified name will receive the event.
Emit a named event. All registered event listeners that listen to the specified name will receive the event.
new observable.Observable()
observable.Observable#on(name: N, f: function)
observable.Observable#once(name: N, f: function)
observable.Observable#off(name: N, f: function)
observable.Observable#emit(name: N, args: Array<any>)
observable.Observable#destroy()
websocket.WebsocketClient#on(name: N, f: function)
websocket.WebsocketClient#once(name: N, f: function)
websocket.WebsocketClient#off(name: N, f: function)
websocket.WebsocketClient#emit(name: N, args: Array<any>)
[lib0/pair] Working with value pairs.
import * as pair from 'lib0/pair.js'
new pair.Pair(left: L, right: R)
pair.create(left: L, right: R): module:pair.Pair<L,R>
pair.createReversed(right: R, left: L): module:pair.Pair<L,R>
pair.forEach(arr: Array<module:pair.Pair<L,R>>, f: function(L, R):any)
pair.map(arr: Array<module:pair.Pair<L,R>>, f: function(L, R):X): Array<X>
[lib0/prng] Fast Pseudo Random Number Generators.
import * as prng from 'lib0/prng.js'
Given a seed a PRNG generates a sequence of numbers that cannot be reasonably predicted. Two PRNGs must generate the same random sequence of numbers if given the same seed.
Create a Xoroshiro128plus Pseudo-Random-Number-Generator. This is the fastest full-period generator passing BigCrush without systematic failures. But there are more PRNGs available in ./PRNG/.
Generates a single random bool.
Generates a random integer with 53 bit resolution.
Generates a random integer with 53 bit resolution.
Generates a random integer with 32 bit resolution.
Generates a random integer with 53 bit resolution.
Generates a random real on [0, 1) with 53 bit resolution.
Generates a random character from char code 32 - 126. I.e. Characters, Numbers, special characters, and Space:
TODO: this function produces invalid runes. Does not cover all of utf16!!
Returns one element of a given array.
prng.DefaultPRNG
prng.create(seed: number): module:prng~PRNG
prng.bool(gen: module:prng~PRNG): Boolean
prng.int53(gen: module:prng~PRNG, min: Number, max: Number): Number
prng.uint53(gen: module:prng~PRNG, min: Number, max: Number): Number
prng.int32(gen: module:prng~PRNG, min: Number, max: Number): Number
prng.uint32(gen: module:prng~PRNG, min: Number, max: Number): Number
prng.int31(gen: module:prng~PRNG, min: Number, max: Number): Number
prng.real53(gen: module:prng~PRNG): Number
prng.char(gen: module:prng~PRNG): string
prng.letter(gen: module:prng~PRNG): string
prng.word(gen: module:prng~PRNG, minLen: number, maxLen: number): string
prng.utf16Rune(gen: module:prng~PRNG): string
prng.utf16String(gen: module:prng~PRNG, maxlen: number)
prng.oneOf(gen: module:prng~PRNG, array: Array<T>): T
prng.uint8Array(gen: module:prng~PRNG, len: number): Uint8Array
prng.uint16Array(gen: module:prng~PRNG, len: number): Uint16Array
prng.uint32Array(gen: module:prng~PRNG, len: number): Uint32Array
[lib0/promise] Utility helpers to work with promises.
import * as promise from 'lib0/promise.js'
Promise.all
wait for all promises in the array to resolve and return the resultChecks if an object is a promise using ducktyping.
Promises are often polyfilled, so it makes sense to add some additional guarantees if the user of this library has some insane environment where global Promise objects are overwritten.
promise.create(f: function(PromiseResolve<T>,function(Error):void):any): Promise<T>
promise.createEmpty(f: function(function():void,function(Error):void):void): Promise<void>
promise.all(arrp: Array<Promise<T>>): Promise<Array<T>>
promise.reject(reason: Error): Promise<never>
promise.resolve(res: T|void): Promise<T|void>
promise.until(timeout: number, check: function():boolean, intervalResolution: number): Promise<void>
promise.wait(timeout: number): Promise<undefined>
promise.isPromise(p: any): boolean
[lib0/queue]
import * as queue from 'lib0/queue.js'
new de#QueueNode()
de#next: module:queue.QueueNode|null
new ueue()
tart: module:queue.QueueNode | null
nd: module:queue.QueueNode | null
(): module:queue.Queue
()
(queue: module:queue.Queue)
()
(queue: module:queue.Queue, n: module:queue.QueueNode)
()
(queue: module:queue.Queue): module:queue.QueueNode | null
()
[lib0/random] Isomorphic module for true random numbers / buffers / uuids.
import * as random from 'lib0/random.js'
Attention: falls back to Math.random if the browser does not support crypto.
random.rand
random.uint32
random.oneOf(arr: Array<T>): T
random.uuidv4
[lib0/set] Utility module to work with sets.
import * as set from 'lib0/set.js'
set.create
set.toArray(set: Set<T>): Array<T>
[lib0/sort] Efficient sort implementations.
import * as sort from 'lib0/sort.js'
Note: These sort implementations were created to compare different sorting algorithms in JavaScript. Don't use them if you don't know what you are doing. Native Array.sort is almost always a better choice.
This algorithm beats Array.prototype.sort in Chrome only with arrays with 10 million entries. In most cases [].sort will do just fine. Make sure to performance test your use-case before you integrate this algorithm.
Note that Chrome's sort is now a stable algorithm (Timsort). Quicksort is not stable.
sort.insertionSort(arr: Array<T>, compare: function(T,T):number): void
sort.quicksort(arr: Array<T>, compare: function(T,T):number): void
[lib0/statistics] Utility helpers for generating statistics.
import * as statistics from 'lib0/statistics.js'
statistics.median(arr: Array<number>): number
statistics.average(arr: Array<number>): number
[lib0/storage] Isomorphic variable storage.
import * as storage from 'lib0/storage.js'
Uses LocalStorage in the browser and falls back to in-memory storage.
This is basically localStorage in browser, or a polyfill in nodejs
A polyfill for
addEventListener('storage', event => {..})
that does nothing if the polyfill is being used.
storage.varStorage
storage.onChange(eventHandler: function({ key: string, newValue: string, oldValue: string }): void)
[lib0/string] Utility module to work with strings.
import * as string from 'lib0/string.js'
Compute the utf8ByteLength
string.fromCharCode
string.fromCodePoint
string.trimLeft(s: string): string
string.fromCamelCase(s: string, separator: string): string
string.utf8ByteLength(str: string): number
string.utf8TextEncoder
string.encodeUtf8
string.decodeUtf8
[lib0/symbol] Utility module to work with EcmaScript Symbols.
import * as symbol from 'lib0/symbol.js'
Return fresh symbol.
symbol.create(): Symbol
symbol.isSymbol(s: any): boolean
[lib0/testing] Testing framework with support for generating tests.
import * as testing from 'lib0/testing.js'
// test.js template for creating a test executable
import { runTests } from 'lib0/testing.js'
import * as log from 'lib0/logging.js'
import * as mod1 from './mod1.test.js'
import * as mod2 from './mod2.test.js'
import { isBrowser, isNode } from 'lib0/environment.js'
if (isBrowser) {
// optional: if this is ran in the browser, attach a virtual console to the dom
log.createVConsole(document.body)
}
runTests({
mod1,
mod2,
}).then(success => {
if (isNode) {
process.exit(success ? 0 : 1)
}
})
// mod1.test.js
/**
* runTests automatically tests all exported functions that start with "test".
* The name of the function should be in camelCase and is used for the logging output.
*
* @param {t.TestCase} tc
*\/
export const testMyFirstTest = tc => {
t.compare({ a: 4 }, { a: 4 }, 'objects are equal')
}
Now you can simply run node test.js
to run your test or run test.js in the browser.
A PRNG for this test case. Use only this PRNG for randomness to make the test case reproducible.
Describe what you are currently testing. The message will be logged.
export const testMyFirstTest = tc => { t.describe('crunching numbers', 'already crunched 4 numbers!') // the optional second argument can describe the state. }
Describe the state of the current computation.
export const testMyFirstTest = tc => { t.info(already crunched 4 numbers!') // the optional second argument can describe the state. }
Group outputs in a collapsible category.
export const testMyFirstTest = tc => { t.group('subtest 1', () => { t.describe('this message is part of a collapsible section') }) await t.groupAsync('subtest async 2', async () => { await someaction() t.describe('this message is part of a collapsible section') }) }
Group outputs in a collapsible category.
export const testMyFirstTest = async tc => { t.group('subtest 1', () => { t.describe('this message is part of a collapsible section') }) await t.groupAsync('subtest async 2', async () => { await someaction() t.describe('this message is part of a collapsible section') }) }
Measure the time that it takes to calculate something.
export const testMyFirstTest = async tc => { t.measureTime('measurement', () => { heavyCalculation() }) await t.groupAsync('async measurement', async () => { await heavyAsyncCalculation() }) }
Measure the time that it takes to calculate something.
export const testMyFirstTest = async tc => { t.measureTimeAsync('measurement', async () => { await heavyCalculation() }) await t.groupAsync('async measurement', async () => { await heavyAsyncCalculation() }) }
testing.extensive
testing.envSeed
new testing.TestCase(moduleName: string, testName: string)
testing.TestCase#moduleName: string
testing.TestCase#testName: string
testing.TestCase#resetSeed()
testing.TestCase#prng: prng.PRNG
testing.repetitionTime
testing.run(moduleName: string, name: string, f: function(module:testing.TestCase):void|Promise<any>, i: number, numberOfTests: number)
testing.describe(description: string, info: string)
testing.info(info: string)
testing.printDom
testing.printCanvas
testing.group(description: string, f: function(void):void)
testing.groupAsync(description: string, f: function(void):Promise<any>)
testing.measureTime(message: string, f: function():void): number
testing.measureTimeAsync(message: string, f: function():Promise<any>): Promise<number>
testing.compareArrays(as: Array<T>, bs: Array<T>, m: string): boolean
testing.compareStrings(a: string, b: string, m: string)
testing.compareObjects(a: Object<K,V>, b: Object<K,V>, m: string)
testing.compare(a: T, b: T, message: string?, customCompare: function(any,T,T,string,any):boolean)
testing.assert(condition: boolean, message: string?)
testing.fails(f: function():void)
testing.runTests(tests: Object<string, Object<string, function(module:testing.TestCase):void|Promise<any>>>)
testing.fail(reason: string)
testing.skip(cond: boolean)
[lib0/time] Utility module to work with time.
import * as time from 'lib0/time.js'
Return current time.
Return current unix time.
Transform time (in ms) to a human readable format. E.g. 1100 => 1.1s. 60s => 1min. .001 => 10μs.
time.getDate(): Date
time.getUnixTime(): number
time.humanizeDuration(d: number): string
[lib0/tree] Red-black-tree implementation.
import * as tree from 'lib0/tree.js'
This is a Red Black Tree implementation
new tree.Tree()
tree.Tree#findNext(id: K)
tree.Tree#findPrev(id: K)
tree.Tree#findNodeWithLowerBound(from: K)
tree.Tree#findNodeWithUpperBound(to: K)
tree.Tree#findSmallestNode(): V
tree.Tree#findWithLowerBound(from: K): V
tree.Tree#findWithUpperBound(to: K): V
tree.Tree#iterate(from: K, from: K, f: K)
tree.Tree#find(id: K): V|null
tree.Tree#findNode(id: K): module:tree~N<V>|null
tree.Tree#delete(id: K)
tree.Tree#put()
[lib0/url] Utility module to work with urls.
import * as url from 'lib0/url.js'
Parse query parameters from an url.
url.decodeQueryParams(url: string): Object<string,string>
url.encodeQueryParams(params: Object<string,string>): string
[lib0/websocket] Tiny websocket connection handler.
import * as websocket from 'lib0/websocket.js'
Implements exponential backoff reconnects, ping/pong, and a nice event system using [lib0/observable].
Whether to connect to other peers or not
new websocket.WebsocketClient(url: string, opts: object, opts.binaryType: 'arraybuffer' | 'blob' | null)
websocket.WebsocketClient#ws: WebSocket?
websocket.WebsocketClient#shouldConnect: boolean
websocket.WebsocketClient#send(message: any)
websocket.WebsocketClient#destroy()
websocket.WebsocketClient#disconnect()
websocket.WebsocketClient#connect()
The MIT License © Kevin Jahns