MVTs
Minimal viable tables (for Julia 0.6)
This is an experimental framework to provide the following feature set:
- Fast, type-stable data tables
- Arbitrary and extensible accelration (and uniqueness) indices applied to a table
- Primitive operations (projections & natural joins) that are acceleration-index aware
- Some acceptable user-level syntax for constructing the primitives
If this proof-of-concept works out, I may apply any learnings to Tabulars.jl, TypedTables.jl, etc.
Current functionality is limited to the basic definitions of rows and tables.
The desired implementation (WIP)
Relation{names} is an abstract type representing a possibly infinite set (or bag) of
named tuples, with the column names names as a tuple of Symbols. Relations are containers of Rows, which
is our local implementation of named tuples. For type stability and disambiguity with,
Symbols, users will interact with names as Label{:name}() which can be constructed simply
by the non-standard string literal l"name" and extracted by labels(relation).
Table{names} is a concrete container which uses column-based storage of Rows, and can
be accessed similarly to a Vector{Row{names}}. Tables can also carry one or more Indexes.
An Index{names} is an acceleration index used to make lookup of values in the specified
columns faster. Some Indexes also represent uniqueness, these are UniqueIndexes.
There are also more abstract Relations. These might represent infinite sets such as
l"a" < l"b" which gives the relation of all possible rows where row[l"a"] < row[l"b"].
Such a relation is not useful on it's own, but can participate in a join to result in a
finite relation.
There are two elementary operations, projections and joins. These operations will be lazy
wherever possible; a set of operations should end in collect to create a Table if
desired.
Projections can remove, add or rename columns, and the appropriate Indexes should be
preserved (and if a column which takes part in an Index is removed, then that Index is
also discarded).
Joins in MVTs.jl are natural inner joins which match up column names and concatenate
rows which are equal on their overlapping portion. A join operation will attempt to use
any available Index to accelerate the join operation. Three-or-more relations may
participate in a join operations.
We will also implement broadcasting so that two tables of the same length can perform operations on their individual rows.