Rook

Rook is a set of middleware and handlers to enable metadata-based routing for Ring web applications.

The intention is to expose a Clojure namespace as a web service resource; there’s a default mapping of HTTP verbs and paths to function names; these can be extended or overridden by metadata on the functions in the namespace.

The end result is that a proper web service resource can be created in very little code.

Rook makes use of Compojure to handle request routing.

Rook works well with Clojure’s core.async to allow you to build richly interdependent resources without blocking all your threads.

Rook is available under the terms of the Apache Software License 2.0.

Rook is available from the Clojars artifact repository as io.aviso:rook. Follow these instructions to configure the dependency in your build tool.

API Documentation

Resource Handler Functions

Rook analyzes the public functions of a namespace to determine which functions are invoked and under which circumstances. The circumstances are a combination of an HTTP verb (GET, POST, etc.) and a Clout route. The route may include keywords. This is called the path specification.

Rook applies a naming convention to functions, so simply naming a function with a particular name implies a certain path specification.

Table 1. Default Handler Functions

Function Name	Verb	Path	Notes
create	POST	/	Create a new resource
destroy	DELETE	/:id	Delete existing resource
edit	GET	/:id/edit	Present HTML form to user to edit existing resource
index	GET	/	List all existing/matching resources
new	GET	/new	Present HTML form to user to create new resource
patch	PATCH	/:id	Modify existing resource; generally implemented as the same as `update`.^[1]
show	GET	/:id	Retrieve single resource by unique id
update	PUT	/:id	Modify existing resource

Rook’s job is to help with routing the incoming request to your functions; it also assists with passing information from the request path or query parameters into your function … the implementation of the function is entirely yours.

You are free to name your functions as you like; in which case, specify the :path-spec metadata on your functions to inform Rook:

(defn active-users
  "List just users who are currently active."
  {:path-spec [:get "/active"]}
  [request]
  ...)

The :path-spec contains two values: the keyword for the verb (this may also be the value :all) and the path to match.

Ring Handlers and Middleware

Rook works within the Ring framework by providing middleware and request handlers, forming a processing pipeline. The section marked namespace-handler in the diagram represents a portion of the overall pipeline constructed by that function.

Customizing and extending Rook is just a matter of understanding how these middleware and handlers work together, and knowing how and when to override or inject your own middleware into the pipeline.

Middleware

Ring is divided into two main pieces: a middleware and a dispatcher.

The middleware analyzes the namespace and the incoming request; it will identify the matching resource handler function, storing the information about it under the :rook request key.

The dispatcher is a Ring request handler that checks for that information; if present it invokes the identified resource handler function. The resource handler function will return a Ring response.

Applications will often add additional middleware in front of the dispatcher; this allows for other concerns to be addressed, such as authentication, logging, error handling, etc.

Argument Resolution

Ring assists with extracting information from the request and provides it as arguments to the invoked resource handler function.

Tip	To avoid confusion, Rook uses the term parameters to refer to query parameters that are part of an HTTP request; it uses arguments to refer to function arguments (which are often called parameters, as well).

Ring uses the name of the argument to identify the value to provide.

First, the argument name is converted to a keyword and a search of the argument resolvers (described shortly) occurs.

If the search is unsuccessful, then a second search occurs, but with dashes in the keyword converted to underscores, e.g., user-id to :user_id.^[2]

It is assumed that standard Ring middleware is in place to convert the :params map from string keys to keyword keys.

Argument resolution is performed by the rook-dispatcher] function, just before invoking the resource handler function. The http://howardlewisship.com/io.aviso/rook/io.aviso.rook.html#var-rook-dispatcher/io.aviso.rook.html#var-default-rook-pipeline[default-rook-pipeline includes rook-dispatcher plus middleware to add any argument resolvers from the function’s :arg-resolvers metadata ^[3] to the list of argument resolvers used by rook-dispatcher.

Warning

If both the namespace and the function define :arg-resolvers metadata, then the function’s list overrides the namespace’s.

Function wrap-with-default-arg-resolvers provides a default set of argument resolvers:

Matching value from the :params key of the request, for access to data provided by query parameters and the request body).
Matching value from the :route-params key of the request, for access to data provided as keywords in the path spec.
Matching value from the :headers key of the request (with the keyword converted to a string, since :headers keys are always lower-case strings).
:params is resolved to the :params map of the Ring request map.
:params* is resolved to the :params map of the Ring request map, but with the keys Clojureized: underscores converted to dashes.
:request is resolved to the Ring request map itself.
:resource-uri is resolved to the base URI for a resource

For :resource-uri, Rook will use information in the Ring request to build a URI that targets a particular resource namespace; this includes the :scheme, :server-name, and :server-port keys, plus the :context key (set by Compojure as part of context routing).

When behind a firewall, the values of the keys may not be valid. Rook will look for a :server-uri request key and use that in preference to building one from :scheme, :server-name:, and +:server-port. You can use middleware to place such a value into the request early in your processing.

Argument resolution can be extended by providing argument resolver functions. An argument resolver function is passed the argument keyword, and the Ring request map and returns the resolved value for the argument.

Important

Arguments may be a map, to leverage map destructing. However, you must always provide the :as key in the map, as that is what Rook will key off of, rather than the argument name as it usually does.

Argument resolvers can fulfill many purposes:

They can validate inputs from the client.
They can convert inputs from strings to other types, such as numbers or dates.
They can provide access to other resources, such as database connection pools.

Argument resolver functions can be specified as metadata directly on the resource handler function; the :arg-resolvers metadata is a sequence of resolvers.

Function arg-resolver-middleware is used to specify additional functions for :arg-resolvers. Argument resolvers added later are considered more specific and so are checked first.

Function build-map-arg-resolver constructs an argument resolver function from a map; It simply returns values from the map.

Function build-fn-arg-resolver constructs an argument resolver function from a map of functions; The functions are selected by the argument keyword, and passed the request.

Tip	Remember that a keyword can act like a function when passed a map, such as the Ring request.

Function request-arg-resolver is an argument resolver that resolves the argument keyword against the Ring request map itself.

arg-resolver-middleware accepts a handler and any number of argument resolvers, allowing them to be easily composed and contributed:

(defn add-standard-resolvers
  [handler conn-pool]
  (arg-resolver-middleware handler
                           (build-map-arg-resolver {:conn-pool conn-pool})
                           request-arg-resolver))

Mapping Namespaces

A typical web service will expose some number of resources; under Rook this means mapping a number of namespaces.

The namespace-handler function is the easy way to do this mapping. It combines compojure.core/context with Rook’s namespace-middleware (which identifies the function to be invoked within the namespace) and default-rook-pipeline.

(routes
  (namespace-handler "/users" 'org.example.resources.users)
  (namespace-handler "/orders" 'org.example.resources.orders))

Important

Rook will require the namespace if it has not already been previously loaded into Clojure.

Remember that the way context works is to match and strip off the prefix, so an incoming GET request for /users/232 will be matched as context /users; Rook will then identify function org.example.resources.users/show with path /:id; ultimately invoking the function with the string value 232 for the id parameter.

In more complicated circumstances, you may have resources in a parent-child relationship. For example, if you were modeling hotels which contain rooms, you might want to access the list of rooms for a particular hotel with the URL /hotels/123/rooms/237:

(routes
  (namespace-handler "/hotels 'org.example.resources.hotels
    (routes
       (namespace-handler "/:hotel-id/rooms" 'org.example.resources.rooms)
       default-rook-pipeline)))

In this example, the first namespace-handler call will match any URL that starts with /hotels. Since that may be a match for the hotels resource itself, or rooms within a specific hotel, the handler for the namespace can’t simply be default-rook-pipeline; instead it is a new route containing a namespace handler, and the default-rook-pipeline for the org.example.resources.hotels namespace.

The nested route matches the :hotel-id symbol from the path; this will be resolved to argument hotel-id in any resource handler function that is invoked in the rooms namespace.

It is important that the default-rook-pipeline both be present, and come last.

If it is missing, then requests for the /hotels URL will be identified by the middleware, but will never be invoked.

If it is present, but comes before the nested namespaces, then a conflict will occur: URLs that should match against the rooms resource will also match against the hotels resource, and since the default-rook-pipeline for the hotels resource is executed first (incorrectly), it will invoke a resource handler function from the hotels namespace.

The namespace middleware always invokes its delegate handler (the request handling function it wraps around), even when no function has been identified. This seems counter-intuitive, but makes sense in the context of the nested resources: for a particular request the hotels namespace may not have a corresponding function to invoke, but the nested rooms namespace may have a matching function.

Also, in the nested resource scenario, the function to invoke may be identified in an outer context, then re-identified, in an inner context, before being invoked.

Writing Rook Middleware

Rook uses the :rook key of the request to store information needed to process requests. With the exception of :arg-resolvers, the values are supplied by the the namespace-middleware function.

:arg-resolvers: List of argument resolvers that apply to any invoked resource handler functions.
:namespace: The symbol identifying the namespace containing the matched function.
:function: The matched function, which will be invoked by default-rook-pipeline.
:metadata: The metadata for the matched function. This is the merged metadata of the function and the namespace (if there are collisions, the function takes precedence).

Rook middleware that fits between namespace-middleware and rook-dispatcher should check for nested request key [:rook :function] to see if a function has been identified.

Validation

Validation is based on Prismatic Schema.

If a function defines :schema metadata, then that is used to validate the request :params. :params contains a merge of query parameters with any data that was submitted in the request body.

Validation assumes that the query parameters keys are converted from strings to keywords (via ring.middleware.keyword-params) and that submitted JSON content is converted to Clojure data using keyword keys (via rink.middleware.format/wrap.restful-format). These filters are part of the standard set of Rook middleware.

Rook performs coercion on the request parameters before validation them and passing them to the next handler. This works best when you define the explicit types as s/Str, s/Int, s/Bool, and use s/enum.

Tip	Use s/Inst to represent time instants (dates that include time). These will be converted from Strings by parsing an ISO-8601 formatted date (yyyy-mm-ddThh:mm:ss.SSSZ).^[4]

You should name your keys for JSON compatibility. By convenience and convention, JSON prefers underscores rather than embedded dashes. Rook’s argument resolvers allow you to use Clojure naming (embedded dashes) in your resource handler functions.

Warning

Schema is, by default, picky: any unexpected key is a failure. Since the Request :params includes arbitrary query parameters, you will usually want to add a mapping of s/Any to s/Any in your top-level schema, to ensure that spurious query parameters do not cause validation errors.

A sample schema might be:

(def index
  {:schema {(s/optional-key :sort_keys)       [(s/enum :first_name :last_name :updated_at)
            (s/optional-key :sort_descending] s/Bool
            (s/optional-key :offset)          s/Int
            (s/optional-key :count)           s/Int
            s/Any                             s/Any}}
  [sort-keys sort-descending offset count]
  ...)

If validation is successful, then processing continues with the coerced request :params. In the above example, if the JSON request body was {"sort_keys":["last_name"]}, then the sort-keys argument will be [:last_name].

If validation is unsuccessful, then a 400 Bad Request response is returned; The body of the response contains a map:

{
  :error "validation-error"
  :failure "..."
}

Warning

What gets reported as the :failure has yet to be worked out.

Sample Server

Below is the minimal setup for a standard Jetty Ring server handling Rook resources.

(ns org.example.server
  (:use
    compojure.core
  (:require
    [ring.adapter.jetty :as jetty]
    [io.aviso.rook :as rook])

(defn start-server
    [port]
    (let [handler (->
                    (routes
                      (namespace-handler "/users" 'org.example.resources.users)
                      (namespace-handler "/orders" 'org.example.resources.orders)
                      (namespace-handler "/hotels 'org.example.resources.hotels
                        (routes
                          (namespace-handler "/:hotel-id/rooms" 'org.example.resources.rooms)
                          rook/default-rook-pipeline)))
                    rook/wrap-with-standard-middleware)])
      (jetty/run-jetty handler {:port port :join? false}))

A more complete example would also configure Twixt for exception reporting, and to (perhaps) provide a client-side application that uses the provided web service.

Async

Rook can be used entirely as a normal set of Ring response handlers and middleware. However, it is even more useful when combined with Clojure’s core.async library.

Rook includes support for an asynchronous pipeline, where processing of a request can occur without blocking any threads (and parts of which may occur in parallel). Async Rook also supports re-entrant requests that bypass the protocol layers; this allows your resource handler functions to easily send loopback requests to other resources within the same server, without needing to encode and decode data, or send HTTP/HTTPs requests, or block threads. This will ensure that your code eats its own dog food by using the same REST APIs it exports, rather than bypassing the APIs to invoke Clojure functions directly.

Finally, Rook includes a client library that makes it very easy to initiate loopback requests and process failure and success responses, again built on top of core.async.

Time will tell just how well this works (its early days yet), but we hope to be able to handle a very large volume of requests very efficiently.

In addition, by leveraging Jetty’s support for continuations, it is easy to create a server that is fully asynchronous end to end: a very small number of request processing threads in Jetty can handle a very large number of concurrent requests, with nearly all the real work taking place in threads managed by go or thread blocks.

More documentation on this is forthcoming.

Warning

When returning a response whose body is an InputStream, it is important to set the Content-Length header to the size of the stream. Failure to do so results in spurious 404 responses to the client.^[5]

1. one function can delegate to the other.

2. The second keyword exists to pragmatically support clients sending JSON, rather than EDN, data; in JavaScript, underscores are easier to wrangle than dashes.

3. Remember that Rook merges function metadata with metadata of the containing namespace

4. This format is compatible with the client-side function Date.toISOString()

5. It is not clear whether this is a bug in Jetty, or related to how Rook’s async support uses Jetty continuations.

tlipski / rook

Rook

Resource Handler Functions

Ring Handlers and Middleware

Middleware

Argument Resolution

Mapping Namespaces

Writing Rook Middleware

Validation

Sample Server

Async

About

Languages