What's this?

A framework to "ergonomically" select custom data input for command line interface (CLI) tools.

The aim is to enrich two-way ⟷ interaction by prepared reference data:

⟶ Human inputs args he remembers (via Tab-auto-completion) in relaxed order.
⟵ Machine feeds back (via Alt+Shift+Q query):
- What args it matched to server data within command schema.
- What else it needs to populate remaining command args.

This broadens applicability of CLI input as a slim alternative to graphical user interface (GUI) competing in convenience for apps especially for developers = "doing more with less".

When your data is instantly and directly queryable, try argcomplete.
When your data is sizeable, user needs performance (indexing), relaxed syntax, keyword-based search, try argrelay.

Interaction overview

User is interrogated based on:

command input schema
data matching already given input

Each command resembles "enum language":

Tokens are tags | labels | keywords from one of the enum sets.
The enum sets are the objects property values within user data.
Fuzzy-search (yet easily predictable) is achieved by:
- relying on rare intersection between enum sets
- allowing unordered args (using priorities to resolve arg type in case enum sets intersect)

Wrapping any command by argrelay:

naturally enables contextual auto-completion in Bash shell (see Tab hotkey below)
provides generic help and navigation (see Alt+Shift+Q hotkey below)
reduces cognitive load with minimalistic enum-based query syntax (matching target executable command line)
maintains small client-side footprint (suitable for resource-constrained terminals)
exposes conveniently browsable data inventory (generic CLI builder)

General dilemma

GUI	CLI
➕ diagrams, images, video	✖️ only via integration with GUI
➖ might be time-consuming for an ad-hoc functionality	➕ always quick dev option (low ceremony)
➖ may not exist early in feature development	➕ likely available early in development
➖ no simple way to store and share GUI output	➕ store and share results as text
➖ repeat steps 500 times? give up!	➕ repeat steps 500 time? loop!
➖ no universal way to reproduce (composite) GUI actions	➕ paste and "replay" commands as text
➖ no universal way to search stored GUI output	➕ `grep`-search results as text
➖ no universal way to compare GUI output	➕ `diff`-compare results as text
➖ no universal way to auto-trigger GUI actions on events	➕ hook commands anyhow (e.g. schedule)
➖ a separate stack (skill set) from backend to contribute to	➕ familiarly dominates backend tools
➖ uses APIs but hardly exposes API to integrate itself	➕ inherent script-ability
➖ limits system access (a layer behind a narrow API)	➕ ultimate control
➕ keyword captions	➖ hardly remembered cryptic `-o` options
➕ point-click actions	➖ increased typing❗
➕ intuitive data-driven interface	➖ interface❓ more like API

While retaining advantages of a CLI tool, argrelay tries to provide those last ➕-s:

intuitive data-driven interface
reduced typing (args auto-reduction)
keyword options (args auto-completion)

As opposed to GUI-demanding approaches like Warp or IDEA terminal,
argrelay survives in basic text modes.

Original use case

Auto-complete commands based on arbitrary data sets,
for example, using metadata for 10s x clusters, 100s x hosts, 1000s x processes, ...
directly from standard shell.

Selecting args directly in shell CLI avoids otherwise error-prone coping-and-pasting via clumsy GUI window switching.

Flexible and responsive lookup required data indexing
(e.g. each Tab-request demands short loading and querying time for context-specific data)
which suggested a split...

Straightforward split: client & server

The performance qualities were achieved by running a standby server with pre-loaded data
(instead of loading this data into each client).

For example, with 1000s of data entries,
even if someone could generate static Bash completion config,
it would take considerable time to load it for every shell instance.

Unlike static | generated | offline index per client, standby server also naturally supports dynamic data updates.

Request hotkeys

Bash:	Server:	Client:
`Alt+Shift+Q`	reports existing and missing input	displays command completion status
`Tab`	suggests options for missing input	lists options to Bash for auto-completion
`Enter`	provides data to invoke a command	executes the command

What's in a name?

CLI for any program is wrapped by argrelay interaction and invoked by the user indirectly.

Eventually, argrelay "relays" command line args (hence, the name)
with associated data around to invoke the program selected by the user:

sequenceDiagram
    autonumber
    participant P as Any program:<br/>user-required<br/>client-side-local
    actor U as <br/>User
    box transparent <br/>argrelay
    participant C as Client
    participant S as Server
    end
    U ->> C: invoke via shell<br/>on hotkeys
    activate C
    C ->> S: "relay" all args
    activate S
    S ->> C: "relay" enriched lookup details
    deactivate S
    C ->> P: "relay" details to invoke
    deactivate C
    activate P
    P ->> U: show results
    deactivate P

Interactive demo

This is a non-intrusive demo (e.g. without permanent changes to ~/.bashrc).

Clone this repo somewhere (@/ is the project root).

Start @/exe/relay_demo.bash (it may take a couple of minutes to start for the first time):

./exe/relay_demo.bash

This sub-shell configures request hotkeys to bind relay_demo command with @/exe/run_argrelay_client:

Interact with relay_demo command (which uses demo test data):

relay_demo goto                 # press `Alt+Shift+Q` to describe available options

relay_demo goto host            # press `Tab` one or multiple times

relay_demo goto host dev        # press Alt+Shift+Q to observe changes in the output

To clean up, exit the sub-shell:
```
exit
```

Beyond the demo

While inside the sub-shell, inspect how auto-completion is configured for relay_demo:
```
complete -p relay_demo
```
See @/logs/relay_demo.bash.log of the background server:
```
less ./logs/relay_demo.bash.log
```
Inspect configs:
- @/conf/argrelay.server.yaml
- @/conf/argrelay.client.json
To reset the demo, remove @/conf:
```
rm conf
```
Script @/exe/relay_demo.bash relies on @/conf being a symlink specifically to @/dst/relay_demo:

If @/conf is absent, it re-creates the symlink with that destination and re-installs everything.
To debug shell scripts, export ARGRELAY_DEBUG with value containing s:
```
export ARGRELAY_DEBUG="s"
./exe/relay_demo.bash
```

What's in the package?

Client to be invoked by Bash hook on every Tab to
send command line arguments to the server.
Server to parse command line and propose values from
pre-loaded data for the argument under the cursor.
Plugins to customize:
- actions the client can run
- objects the server can search
- grammar the command line can have
Interfaces to bind these all together.
Bootstrap process to init the environment and maintain it.
Demo example to start from.
Testing support and coverage.

Focus: CLI search and data-assisted completion

GUI-s are secondary for argrelay's purpose because
GUI-s do not have the restrictions CLI-s have:

Technically, the server can handle requests from anywhere (GUI).
But primary API-s are feature-tailored to support CLI (because everyone does GUI).

show example

For example, in GUI-s, typing a query into a search bar may easily be accompanied by
(1) a separate (from the search bar) window area
(2) with individually selectable
(3) full-text-search results
(4) populated **async-ly** with typing.

In CLI-s, grep does (3) full-text-search, but slow and what about the rest (1), (2), (4)?

To facilitate selection of results,
catalogue-like navigation via structured search (rather than full-text-search) with auto-completion
seems the answer.

Nevertheless, GUI can also benefit from minimalist single line structured search queries.

Data backend

There are two options at the moment - both using MongoDB API:

Category	`mongomock` (default)	`pymongo`
Data set size:	practical convenience limit ~ 10K objects	tested with ~ 1M objects
Pro:	nothing else to install	no practical data set size limit found (yet) for `argrelay` intended use cases
Con:	understandably, does not meet performance requirements for large data sets	require some knowledge of MongoDB, additional setup, additional running processes

Quantitative comparison tables between the two can be seen in docstring for DistinctValuesQuery enum.

pymongo connects to a running MongoDB instance which has to be configured in
argrelay.server.yaml under mongo_config and mongomock should be disabled:

-    use_mongomock: True
+    use_mongomock: False

Full picture

sequenceDiagram
    autonumber
    actor U as <br/>User
    participant B as Bash
    participant P as Any program:<br/>user-required<br/>client-side-local
    box transparent <br/>argrelay
    participant C as Client
    participant S as Server
    participant DB as Data backend<br/>(internal or external)
    end
    participant DS as Data sources
    DS ->> S: load data
    activate S
    S ->> DB: load data
    deactivate S
    Note over S: <br/>stand-by<br/>
    U ->> B: enter command and use hotkeys
    B ->> C: invoke
    activate C
    C ->> S: "relay" all args
    activate S
    S ->> DB: query request
    activate DB
    DB ->> S: query result
    deactivate DB
    S ->> C: "relay" enriched lookup details
    deactivate S
    Note over C: next steps depend on hotkeys
    C ->> U: show results
    C ->> P: "relay" details to invoke
    deactivate C
    activate P
    P ->> U: show results
    deactivate P

Feedback

Feel free to raise issues or discussions.

argrelay / argrelay