mabhub / pivotql

PivotQL is a generic query language that produce an unist AST.

PivotQL (Pivot Query Language)

Lets you write a simple query and run it against any datastore, like ElasticSearch and/or Mapbox filter.

PivotQL parses a simple query language into an Abstract Syntax Tree implementing the unist spec that can then be used to compile a query for a given datastore.

This is useful if you want a level of abstraction between your queries and how your data is actually stored/searched. You can also use PivotQL to execute the same query against multiple datastores at the same time.

For example:

import {parse} from 'pivotql-parser-expression';
import {compiler as esCompiler} from 'pivotql-compiler-elasticsearch';
import {compiler as mapboxCompiler} from 'pivotql-compiler-mapbox';

// parse a pivotql query into an AST
const queryAST = parse( '( height <= 20 or ( favorites.color == "green" and height != 25 ) ) and firstname in ["john", "doug"]' );

// using that AST, compile an elasticsearch query
const esQuery = esCompiler( queryAST );
console.log( util.inspect( esQuery, { depth: null } ) );

// using the same AST, compile an mabox filter query
const mapboxQuery = mapboxCompiler( queryAST );
console.log( util.inspect( mapboxQuery, { depth: null } ) );

Which produces the following AST:

{
"type":"&&",
"children":[
  {"type":"EXPRESSION","children":[
    {"type":"||","children":[
      {"type":"<=","children":[
        {"type":"SYMBOL","value":"height"},{"type":"NUMBER","value":20}]},
        {"type":"EXPRESSION","children":[
          {"type":"&&","children":[
            {"type":"==","children":[
              {"type":"SYMBOL","value":"favorites.color"},{"type":"STRING","value":"green"}]},
              {"type":"!=","children":[
                {"type":"SYMBOL","value":"height"},{"type":"NUMBER","value":25}]}]}]}]}]},
    {"type":"IN","children":[
      {"type":"SYMBOL","value":"firstname"},{"type":"ARRAY","children":[
        {"type":"STRING","value":"john"},{"type":"STRING","value":"doug"}]}]}]
}

Using that AST, you can generate queries for various datastores.

For ElasticSearch, the query is:

{
"query":{"filter":[
  {"bool":{"must":[
    {"bool":{"should":[
      {"range":{"height":{"lte":20}}},{"bool":{"must":[
        {"term":{"favorites.color":"green"}},
        {"bool":{"must_not":{"term":{"height":25}}}}]}}]}},
  {"bool":{"must":{"bool":{"should":[
    {"match":{"firstname":"john"}},{"match":{"firstname":"doug"}}]}}}}]}}]}
}

and produces this for mapbox:

{
"filter":[
  "all",[
    "any",["<=",["get","height"],20],[
      "all",["==",["get","favorites.color"],"green"],["!=",["get","height"],25]]],
    ["in",["get","firstname"],["literal",["john","doug"]]]]
}

Available parsers

• Simple expression
• !! Your parser here! Submissions are welcome! !!

Available Compilers

• Elastic Search
• Mapbox
• MongoDB
• Javascript
• !! Your compiler here! Submissions are welcome! !!

Development

This repository contains many packages in the packages/ directory. You need npm > v7.14 with workspaces features to be able to execute the next commands:

npm ci
npm build # Generate parser with Jison

And launch test:

npm run test

Credits

This was inspired by UniQL, inspired itself by FiltrES which was in turn inspired by Filtrex