HTMX Filter Tree Aggregations PoC

PoC demonstrating a dynamic filtering form with HTMX for a SIP/Mail-Server applicance.

The backend is implemented in Common Lisp, using the 3rd party systems cl-who (HTML generation), esrap (Packrat parsing), hunchentoot (HTTP server), cl-sqlite (SQL database).

Installation

Install SBCL (Other Common Lisp implementations such as Clozure or ECL may work, but are untested).
Install the ocicl package manager.

Deployment

Run ocicl install to install all dependencies.

Starting

Run the following command:

sbcl --disable-debugger \
     --eval '(pushnew (uiop:getcwd) asdf:*central-registry*)' \
     --eval '(asdf:load-system :xfiltertree-server)' \
     --eval '(hunchentoot:start xfiltertree-server:*acceptor*)' \
     --eval '(sleep #xffffffff)'

Then access the server at http://localhost:8080

Systems

WEBSTR

Provides WEBSTR:ESCAPE that hex-escapes strings for safe usage in HTML/CSS. All non-alphanumeric characters C are translated to _XX_ where XX = hex(C).

WEBSTR:UNESCAPE unescapes any string produced by WEBSTR:ESCAPE so that (WEBSTR:UNESCAPE (WEBSTR:ESCAPE) S) ≣ S.

EQVALG

Equivalence algebra system. Provides EQUALITY MEMBERSHIP and COJUNCTION types to represent terms in AND conjunctions of equalities. COLUMN is provided as an abstract variable type representing a table column.

A COALESCE(X,Y) operator is provided to combine terms X and Y, which can be instances of any of the aforementioned term types.

FQL

Uses EQVALG.

Parser & grammar definition for a simple equivalency expression language. (FQL:PARSE-FILTER (STRING)) parses STRING into an EQVALG term object.

table.column=x is parsed into an EQVALG:EQUALITY object.

Subtyping clauses are also supported as a special case. table[type=t].column=x is parsed into (EQVALG:CONJUNCTION (EQVALG:EQUALITY (EQVALG:COLUMN table type) t) (EQVALG:EQUALITY (EQVALG:COLUMN table column) x)).

XFILTERTREE

Defines the classes NODE and AGGREGATION where AGGREGATION extends NODE.

Each NODE has an ID and a list of CHILDREN nodes.

An aggregation has an additional associative list BINS which is a list of the form ((BIN-ID (AGG-1) (AGG-2) …) …). Note that the CDR of (AGG-N) is mutable by convention and may be set by processing functions to the aggregation count.

A DYNAMIC node extends AGGREGATION with QUERIER and SEARCHER properties, which are conventionally set to URIs that implement aggregation querying and DWIM searching for the dynamic node's resource types.

XFILTERTREE-EQVALG

Uses XFILTERTREE, EQVALG.

Provides (XFILTERTREE-EQVALG:CONSTRAIN (TREE CONSTRAINTS) that mutates TREE by coalescing all its XFILTERTREE:AGGREGATION nodes' aggregation predicates with the given constraints.

Provides (XFILTERTEE-EQVALG:EXTEND (TREE DYNAMIC)) that mutates TREE by extending each XFILTERTREE:DYNAMIC node within the TREE with aggregation clauses from DYNAMIC that are deemed compatible with it. A compatible aggregation clause has a subject table that matches the DYNAMIC node's ID.

XFILTERTREE-HTML

Uses XFILTERTREE, WEBSTR.

HTML/Hypermedia rendering system for XFILTERTREE trees.

(XFILTERTREE-HTML:HTMLIZE (NODE)) returns a string containing a htmx-enabled hypermedia HTML form given a XFILTERTREE:NODE instance.

XFILTERTREE-HTML:*TRANSLATE* is a dynamic parameter used to stringify NODE IDs and aggregation bin IDs into a string representation. This is used to facilitate conversion of custom ID types. This defaults to #'IDENTITY if not otherwise bound.

XFILTERTREE:AGGREGATION nodes are transformed into <fieldset> nodes containing checkbox inputs, allowing for selection of aggregation bin. One checkbox input is generated for each bin. The value=… HTML attribute is set to be equivalent to the WEBSTR:ESCAPE-ed *TRANSLATE*-transformed bin ID, and the HTML name=… attribute is set to clause.

XFILTERTREE-HTML:*FORM-POST* should be bound to the form submission POST endpoint, and defaults to / if not bound otherwise.

If XFILTERTREE-HTML:*FORM-UPDATE* is bound to T (or any generalized true value), XFILTERTREE-HTML:HTMLIZE will generate a DOM update payload, where tags may have an additional hx-oob-swap property facilitating htmx-driven updates of bin counts with minimal rerendering.

XFILTERTREE-BOM

Uses XFILTERTREE, EQVALG.

Defines factories for creating XFILTERTREE nodes.

XFILTERTREE:MAKE-TREE is a factory that generates a predefined filter tree.

XFILTERTREE-SQL

Uses XFILTERTREE, EQVALG-SQL.

Generates a filter node tree from a sqlite3 database using EQVALG-SQL.

Provides (XFILTERTREE:COMPUTE-AGGREGATIONS (TREE)) which mutates the CDR of aggregation pairs in all AGGREGATION nodes in tree, setting the aggregation count for each bin by querying an SQL database.

XFILTERTREE-SQL:*DB* should be bound to the sqlite3 database file (this defaults to db.sqlite3).

XFILTERTREE-SERVER

Uses FQL, XFILTERTREE-EQVALG, XFILTERTREE-BOM, XFILTERTREE-HTML, XFILTERTREE-SQL.

Integrates the systems XFILTERTREE-HTML, XFILTERTREE-SQL and HUNCHENTOOT to provide a filter tree server.

XFILTERTREE-SERVER:*ACCEPTOR* is the hunchentoot acceptor, with all routes defined. By default, it's configured to listen on localhost:8080.

The server may be interactively started with (HUNCHENTOOT:START XFILTERTREE-SERVER:*ACCEPTOR*), and stopped with (HUNCHENTOOT:STOP XFILTERTREE-SERVER:*ACCEPTOR*).

fmqa / xfilter-hx