A Clojure library that provides:
- A web framework based around Jetty and Ring
- Utility scripts for managing databases, email, logging, SSL certs, systemd services, and more
- Utility namespaces with functions for common CLI and web server programming tasks
- Add the library’s coordinates to the
:deps
map in your project’sdeps.edn
file:sig-gis/triangulum {:git/url "https://github.com/sig-gis/triangulum" :git/sha "REPLACE-WITH-CURRENT-SHA"}
If you want to pull Triangulum into a Babashka script, you can do so with
babashka.deps
as follows:(require '[babashka.deps :refer [add-deps]]) (add-deps '{:deps {sig-gis/triangulum {:git/url "https://github.com/sig-gis/triangulum" :git/sha "REPLACE-WITH-CURRENT-SHA"}}})
- Define one or more aliases for Triangulum’s utility scripts in the
:aliases
map in your project’sdeps.edn file
.NOTE: You can find examples of these aliases in the
deps.example.edn
file in the root directory of this repository. - Configure the Triangulum features you want to use in a
config.edn
file in the root directory of your repository.Two syntaxes are provided for this file, one with nested keywords and one with namespaced keywords. You can see examples of the available features in
config.nested-example.edn
andconfig.namespaced-example.edn
within this repository.
To launch the test suite, run:
clojure -M:test
This namespace provides a batteries-included Ring Jetty web server, an nrepl server for connecting to the live web application, a timestamped file logging system, and worker functions for non-HTTP-related tasks.
- In config.edn:
:server {:http-port 8080 :https-port 8443 ; Only if you also include the keystore fields below :nrepl-bind 127.0.0.1 ; Must be paired with either :nrepl or :cider-repl below :nrepl-port 5555 ; Must be paired with either :nrepl or :cider-repl below :nrepl false ; For a vanilla nrepl :cider-nrepl true ; If your editor supports CIDER middleware :mode "dev" ; or prod :log-dir "logs" ; or "" for stdout :handler product-ns.routing/handler :workers {:scheduler {:start product-ns.jobs/start-scheduled-jobs! :stop product-ns.jobs/stop-scheduled-jobs!}} :response-type :json ; #{:json :edn :transit} :session-key "changeme12345678" :bad-tokens #{".php"} ; Reject URLs containing these strings :keystore-file "keystore.pkcs12" ; Contains your SSL certificate(s) :keystore-type "pkcs12" :keystore-password "foobar"}
- At the command line:
Start the server:
clojure -M:server start
Stop the server (requires
:nrepl
or:cider-nrepl
):clojure -M:server stop
Reload your namespaces into the running JVM (requires
:nrepl
or:cider-nrepl
):clojure -M:server reload
NOTES:
- To use
:https-port
, you must also specify:keystore-file
,:keystore-type
, and:keystore-password
. - You may not use both
:nrepl
and:cider-nrepl
at the same time. - The
:start
functions for all:workers
will be run before the web server is started. - The
:stop
functions for all:workers
will be run after the web server is stopped. They will receive the outputs of the:start
functions. - Before starting the server, you must set
:handler
to a namespace-qualified symbol bound to a Ring handler function. - If you set
:mode
to “dev”, then thewrap-reload
middleware will be added to your handler function. This will automatically reload all of your namespaces into the running JVM on each HTTP request.
- To use
The triangulum.handler
namespace provides core request handling and
middleware composition for Triangulum applications. It sets up a Ring
handler stack that includes various middlewares, such as
request/response logging, exception handling, and request parameter
parsing. Optional middlewares like wrap-ssl-redirect
and
wrap-reload
can be applied based on your config.edn
settings.
If you need to provide a symbol that is bound to a handler function to
figwheel-main
, you can use triangulum.handler/development-app
to
load in your project’s handler function from config.edn
with the
standard Triangulum middlewares added.
This namespace provides functions for rendering pages and handling resources in Triangulum. It defines functions for reading asset files, generating HTML, and handling various types of responses.
- Require the namespace in your project.
- Use
render-page
to generate a handler that will return an HTML response with a standard template for React/Reagent web apps.
(ns my-app.views
(:require [triangulum.views :refer [render-page]]))
(def my-page-handler (render-page "/my-page"))
[uri]
Returns a function that takes a request and generates the HTML for the specified URI using the request’s parameters and session data. The generated HTML includes the necessary head and body sections.
Example usage: (def my-page (render-page “/my-page”))
In JavaScript projects, we assign the relative path (from the project root)
to the main component JSX file to the :js-init
key. This file should export
a function called pageInit
that expects two arguments: params
and session
.
You only need to set this key for the development mode to work,
which enables Vite hot reload. In production, we rely on a manifest file
generated by the bundling process to find the entry point. However, we still
need to define pageInit
and export it in the main entry point file.
In ClojureScript projects, we need to assign the namespaced symbol of the init
function to the :cljs-init
key, which accepts params
and session
as
arguments, for both production and development environments.
The session
map will also contain the :client-keys
that were added in
Triangulum’s config.edn
.
;; nested config
{:app {:client-keys {:token "client-token" }}}
;; namespaced config
{:triangulum.views/client-keys {:token "client-token" }}}
You can provide :tags-url
, which is a url to the git tags page of
your repository. Triangulum will extract all tags beginning with
“prod”, sort them lexicographically, and return the last entry. If you
use tags of the form “prod-YYYY.MM.DD-HASH”, then this will return the
one with the latest date.
This tag label will be passed to the browser code in the :session
map under the :versionDeployed
key.
To set up the folder and file structure for use with build-db
, use the following directory structure:
src/
|___clj/
| |___<project namespace>
|
|___cljs/
| |___<project namespace>
|
|___sql/
|___create_db.sql
|___changes/
|___default_data/
|___dev_data/
|___functions/
|___tables/
You may also run this command in your project root directory:
mkdir -p src/sql/{changes,default_data,dev_data,functions,tables}
Postgresql needs to be installed on the machine that will be hosting this website. This installation task is system specific and is beyond the scope of this README, so please follow the instructions for your operating system and Postgresql version. However, please ensure that the database server’s superuser account is named “postgres” and that you know its database connection password before proceeding.
Once the Postgresql database server is running on your machine, you
should navigate to the top level directory (i.e., the directory
containing this README) and add the following alias to your deps.edn
file:
{:aliases {:build-db {:main-opts ["-m" "triangulum.build-db"]}}}
Then run the database build command as follows:
clojure -M:build-db build-all -d database [-u user] [-p admin password]
This will call ./src/sql/create_db.sql
, stored in the individual project
repository. A variable database
is set for the command line call to
create_db.sql. This allows your project to generate the project database
with a different name, depending on your deployment. To use this variable
type :database
in create_db.sql
where needed. You can check out
Collect Earth Online
to view an example.
A handy use of the build-db
command is to backup and restore your database.
Calling
clojure -M:build-db backup -f somefile.dump
will create a .dump
backup file using pg_dump
.
To restore your database from a .dump
file you will need a .dump
file
containg a copy of a database downloaded locally. Assuming you have a copy of
a database, you can then run:
clojure -M:build-db restore -f somefile.dump
This will copy the database from the .dump
file into your local Postgres
database of the same name as the one in the .dump
file. Note that you will be
prompted with a password after running this command. You should enter the
Postgres master password that you first created when running Postgres after
installing. Depending on the size of your .dump
file, this command may take a
couple of minutes. Note that if you are working on a development branch and your
.dump
file contains a copy of a production database you may also need to apply
some of the SQL changes from the ./sql/changes
directory. Assuming your
database doesn’t have any of the change files on development applied to it,
you can apply all of them at once using the following command:
for filename in ./src/sql/changes/*.sql; do psql -U <db-name> -f $filename; done
triangulum.build-db can also be configured through config.edn. It uses the same configuration as triangulum.database (see above).
To make organizing an application’s configurations simpler, create a
config.edn
file in the project’s root directory. The file is just a hashmap that is similar to:
;; config.edn
{:database {:host "localhost"
:port 5432
:dbname "dbname"
:user "user"
:password "super-secret-password"}
:mail {:host "smtp.gmail.com"
:user "test@example.com"
:pass "3492734923742"
:port 587}
:server {:host "smtp.gmail.com"
:user ""
:pass ""
:tls true
:port 587
:base-url "https://my.domain/"
:auto-validate? false}
...}
You can find an up-to-date example in config.nested-example.edn
file. It can be used as a configuration template for your project.
Add config.edn to your .gitignore
file to keep sensitive information out of
the git history.
To validate the config.edn file, run:
clojure -M:config validate [-f FILE]
To retrieve a configuration, use get-config
. You can supply nested
configuration keys as follows:
(triangulum.config/get-config :database) ;; -> {:user "triangulum" :pass "..."}
(triangulum.config/get-config :database :user) ;; -> "triangulum"
(triangulum.config/get-config :server) ;; -> {:http-port 8080 :mode "dev"}
(triangulum.config/get-config :server :http-port) ;; -> 8080
See each section below for an example configuration if one is required for use.
To build a JAR file from your repository and deploy it to clojars.org, run:
env CLOJARS_USERNAME=$YOUR_USERNAME CLOJARS_PASSWORD=$YOUR_CLOJARS_TOKEN clojure -M:deploy $GROUP_ID $ARTIFACT_ID
NOTE: As of 2020-06-27, Clojars will no longer accept your Clojars password when deploying. You will have to use a token instead. Please read more about this here
If you have not already created a SSL certificate, you must start a server
without a https port specified. (e.g. clojure -M:run-server
).
Add the following alias to your deps.edn
file:
{:aliases {:https {:main-opts ["-m" "triangulum.https"]}}}
To automatically create an SSL certificate signed by Let’s Encrypt, simply run the following command from your shell:
sudo clojure -M:https certbot-init -d mydomain.com [-p certbot-dir] [--cert-only]
The certbot creation process will run automatically and silently.
Note: If your certbot installation stores its config files in a directory other than /etc/letsencrypt, you should specify it with the optional certbot-dir argument to certbot-init.
Certbot runs as a background task every 12 hours and will renew any
certificate that is set to expire in 30 days or less. Each time the
certificate is renewed, any script in /etc/letsencrypt/renewal-hooks/deploy
will be run automatically to repackage the updated certificate into the correct
format.
If certbot runs successfully and –cert-only is not specified, then a shell script
[mydomain].sh will be created in the certbot deploy hooks folder.
This script will run clojure -M:https package-cert
. Scripts in this folder will
run automatically when a new certificate is created.
While there should be no need to do so, if you ever want to perform this repackaging step manually, simply run this command from your shell:
sudo clojure -M:https package-cert -d mydomain.com [-p certbot-dir]
Create a shell script in /etc/letsencrypt/renewal-hooks/deploy
and update permissions.
sudo nano /etc/letsencrypt/renewal-hooks/deploy/custom.sh
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/custom.sh
To make sure your application starts up on system reboot, you can use
Triangulum to create a systemd user .service
file by adding the following to
your :aliases
section in the deps.edn
file:
{:aliases {:systemd {:main-opts ["-m" "triangulum.systemd"]}}}
Modify your app code to call (triangulum.notify/ready!)
after all of your
application’s services are started:
(ns <app>.server
(:require [triangulum.notify :as notify]))
...
(defn app-start []
(reset! db (jdbc/connect!))
(reset! queues (q/start!))
(reset! server (ring/start-server!)
(when (notify/available?) (notify/ready!))))
And then run:
clojure -M:systemd enable -r $REPO [-p $HTTP_PORT] [-P $HTTPS_PORT] [-d $REPO_DIRECTORY] [-A $EXTRA_ALIASES]
This will install a file named cljweb-<repo>.service
into the
~~/.config/systemd/user/~ directory, reload the systemctl
daemon,
and enable your service. By default, the current directory will be
used in the service as the working directory. To supply an
alternative, you can use -d
. This will look for a Clojure project in
that directory.
The server will always be started using clojure -M:server start
unless the --extra-aliases
option is passed. In that case, it will
run with clojure -M${EXTRA_ALIASES}:server start
.
To enable your user services to start on system reboot, you will need to run:
sudo loginctl enable-linger "$USER"
Now your service will be enabled at startup. You can also start, stop, and restart your service with the following commands:
clojure -M:systemd start -r <REPO>
clojure -M:systemd stop -r <REPO>
clojure -M:systemd restart -r <REPO>
The triangulum.cli namespace provides a command-line interface (CLI) for Triangulum applications. It includes functions for parsing command-line options, displaying usage information, and checking for errors in the provided arguments.
Use get-cli-options to parse command-line arguments and return the user’s options.
(def cli-options {...})
(def cli-actions {...})
(def alias-str "...")
(get-cli-options command-line-args cli-options cli-actions alias-str)
Takes the command-line arguments, a map of CLI options, a map of CLI actions, an alias string, and an optional config map. Checks for valid CLI calls and returns the user’s options.
The triangulum.errors namespace provides error handling utilities for the Triangulum application. It includes functions and macros to handle exceptions and log errors.
Takes a message string as input and throws an exception with the provided message.
(init-throw "Error: Invalid input")
Takes a function try-fn and a message string as input. Executes the function and, if it throws an exception, catches the exception, logs the error, and then throws an exception with the augmented input message.
(try-catch-throw (fn [] (throw (ex-info "Initial error" {}))) "Augmented error message")
Public Macros
Catches any exceptions thrown within its body and returns nil if an exception occurs. If no exception occurs, it returns the result of the body’s evaluation.
(nil-on-error (/ 1 0)) ; Returns nil
(nil-on-error (+ 2 3)) ; Returns 5
You can set :response-type
to configure the data-response
function’s default return type (:json
, :edn
, :transit
).
The triangulum.utils
namespace provides a collection of utility functions for various purposes, such as text parsing, shell command execution, response building, and operations on maps and namespaces.
Converts a kebab-cased string to a snake_cased string.
Converts a kebab-cased string to a camelCased string.
Formats a string with placeholders (e.g., “%s”) replaced by the provided arguments.
Splits a string into an array for use with clojure.java.shell/sh
.
Appends a specified string to the end of another string, if it is not already there.
Removes a specified string from the end of another string, if it is there.
A wrapper around babashka.process/shell
that logs the output and errors. Accepts an optional opts map as the first argument, followed by the command and its arguments. The :log?
key in the opts map can be used to control logging (default is true).
Runs a set of bash commands in a specified directory and environment. Parses the output, creating an array as described in parse-as-sh-cmd
.
Use ‘triangulum.response/data-response’ instead. Creates a response object with a specified body, status, content type, and session.
Operations on Maps
Applies a function to each MapEntry of a map, returning a new map.
Filters a map based on a predicate applied to each MapEntry, returning a new map.
Reverses the key-value pairs in a given map.
Checks if the keys of one map are a subset of another map’s keys, including nested maps.
Attempts to require a namespace-qualified symbol’s namespace and resolve the symbol within that namespace to a value.
Recursively delete all files and directories under the given directory.Traverses the directory tree in reverse depth-first order.
The triangulum.type-conversion namespace provides a collection of functions for converting between different data types and formats, including conversions between numbers, booleans, JSON, and PostgreSQL data types.
Converts a value to a Java Integer. Default value for failed conversion is -1.
Converts a value to a Java Long. Default value for failed conversion is -1.
Converts a value to a Java Float. Default value for failed conversion is -1.0. Note that Postgres real is equivalent to Java Float.
Converts a value to a Java Double. Default value for failed conversion is -1.0. Note that Postgres float is equivalent to Java Double.
Converts a value to a Java Boolean. Default value for failed conversion is false.
Converts a JSON string to its Clojure equivalent.
Converts a PostgreSQL jsonb object to a JSON string.
Converts a PostgreSQL jsonb object to its Clojure equivalent.
Converts a Clojure value to a JSON string.
Converts a string to a PostgreSQL object of a specified type.
Converts a JSON string to a PostgreSQL jsonb object.
Converts a Clojure value to a PostgreSQL jsonb object.
The triangulum.sockets namespace provides functionality for creating and managing client and server sockets. It includes functions for opening and checking socket connections, sending messages to the server, and starting/stopping socket servers with custom request handlers. This namespace enables communication between distributed systems and allows you to implement networked applications.
Checks if the socket at the specified host and port is open.
Attempts to send a socket message. Returns :success
if successful.
Stops the running socket server.
Starts a socket server at the specified port with a custom request handler.
The triangulum.notify namespace provides functions to interact with systemd for process management and notifications. It utilizes the SDNotify Java library to send notifications and check the availability of the current process. The functions in this namespace allow you to check if the process is managed by systemd, send “ready,” “reloading,” and “stopping” messages, and send custom status messages. These functions can be helpful when integrating your application with systemd for better process supervision and management.
Checks if this process is a process managed by systemd.
Sends a ready message to systemd. Systemd file must include Type=notify to be used.
Sends a reloading message to systemd. Must call send-notify! once reloading has been completed.
Sends a stopping message to systemd.
Sends a custom status message to systemd. (e.g. (send-status! "READY=1")).
Triangulum provides some functionality for sending email from an SMTP server. Given the configuration inside :mail
. :base-url
is used to configure the host url, used when sending links in emails. :auto-validate
can be used in development mode, for example, to skip sending emails, which has to be configured.
To send a message to the logger use log
or log-str
. log
can take an
optional argument to specify not default behavior. The default values are
shown below. log-str
always uses the default values.
(log "Hello world" {:newline? true :pprint? false :force-stdout? false})
(log-str "Hello" "world")
By default the above will log to standard out. If you would like to have the system log to YYYY-DD-MM.log, set a log path. You can either specify a path relative to the toplevel directory of the main project repository or an absolute path on your filesystem. The logger will keep the 10 most recent logs (where a new log is created every day at midnight). To stop the logging server set path to “”.
(set-log-path "logs")
(set-log-path "")
To use triangulum.database
, first add your database connection
configurations to a config.edn
file in your project’s root directory.
For example:
;; config.edn
{:database {:host "localhost"
:port 5432
:dbname "pyregence"
:user "pyregence"
:password "pyregence"}}
To run a postgres sql command use call-sql
. Currently call-sql
only works with postgres. With the second parameter can be an
optional settings map (default values shown below).
(call-sql "function" {:log? true :use-vec? false} "param1" "param2" ... "paramN")
To run a sqllite3 sql command use call-sqlite
. An existing sqllite3 database
must be provided.
(call-sqlite "select * from table" "path/db-file")
To insert new rows or update existing rows use insert-rows!
and
update-rows!
. If fields are not provided, the first row will be assumed to
be the field names.
(insert-rows! table-name rows-vector fields-map)
(update-rows! table-name rows-vector column-to-update fields-map)
The triangulum.worker
namespace is responsible for the management of worker lifecycle within the :server
context, specifically those defined under the :workers
key. This namespace furnishes functions to initiate and terminate workers, maintaining their current state within an atom.
Starts a set of workers based on the provided configuration. The workers parameter can be either a map (for nested workers) or a vector (for namespaced workers).
For nested workers, the map keys are worker names and values are maps with :start (a symbol representing the start function) and :stop keys. The start function is called to start the worker.
For namespaced workers, the vector elements are maps with ::name (the worker name), ::start (a symbol representing the start function), and ::stop keys. The start function is called to start each worker.
Arguments:
- workers: a map or vector representing the workers to be started.
Stops a set of currently running workers. The workers to stop are determined based on the current state of the `workers` atom. If the `workers` atom contains a map, it’s assumed to be holding nested workers. If it contains a vector, it’s assumed to be holding namespaced workers.
The stop function is called with the value to stop each worker.
There are no arguments for this function.
Copyright © 2021-2023 Spatial Informatics Group, LLC.
Triangulum is distributed by Spatial Informatics Group, LLC. under the terms of the Eclipse Public License version 2.0 (EPLv2). See LICENSE.txt in this directory for more information.