Muuntaja

Clojure library for fast http format negotiation, encoding and decoding. Standalone library, but ships with adapters for ring (async) middleware & Pedestal-style interceptors. Explicit & extendable, supporting out-of-the-box JSON, EDN and Transit (both JSON & Msgpack). Ships with optional adapters for MessagePack and YAML.

Based on ring-middleware-format, but a complete rewrite (and up to 30x faster).

Rationale

explicit configuration
fast with good defaults
extendable & pluggable: new formats, behavior
typed exceptions - caught elsewhere
support runtime docs (like swagger) & inspection (negotiation results)
support runtime configuration (negotiation overrides)

Modules

metosin/muuntaja - the core abstractions + Jsonista JSON, application/x-www-form-urlencoded, EDN and Transit formats
metosin/muuntaja-cheshire - optional Cheshire JSON format
metosin/muuntaja-form - optional application/x-www-form-urlencoded formatter using ring-codec
metosin/muuntaja-msgpack - Messagepack format
metosin/muuntaja-yaml - YAML format

Posts

Muuntaja, a boring library everyone should use

Check the docs on cljdoc.org for detailed API documentation as well as more guides on how to use Muuntaja.

Latest version

[metosin/muuntaja "0.6.7"]

Optionally, the parts can be required separately:

[metosin/muuntaja-form "0.6.7"]
[metosin/muuntaja-cheshire "0.6.7"]
[metosin/muuntaja-msgpack "0.6.7"]
[metosin/muuntaja-yaml "0.6.7"]

Muuntaja requires Java 1.8+

Quickstart

Standalone

Use default Muuntaja instance to encode & decode JSON:

(require '[muuntaja.core :as m])

(->> {:kikka 42}
     (m/encode "application/json"))
; => #object[java.io.ByteArrayInputStream]

(->> {:kikka 42}
     (m/encode "application/json")
     slurp)
; => "{\"kikka\":42}"

(->> {:kikka 42}
     (m/encode "application/json")
     (m/decode "application/json"))
; => {:kikka 42}

Ring

Automatic decoding of request body and response body encoding based on Content-Type, Accept and Accept-Charset headers:

(require '[muuntaja.middleware :as middleware])

(defn echo [request]
  {:status 200
   :body (:body-params request)})

; with defaults
(def app (middleware/wrap-format echo))

(def request
  {:headers
   {"content-type" "application/edn"
    "accept" "application/transit+json"}
   :body "{:kikka 42}"})
   
(app request)
; {:status 200,
;  :body #object[java.io.ByteArrayInputStream]
;  :headers {"Content-Type" "application/transit+json; charset=utf-8"}}

Automatic decoding of response body based on Content-Type header:

(-> request app m/decode-response-body)
; {:kikka 42}

There is a more detailed Ring guide too. See also differences to ring-middleware-format & ring-json.

Interceptors

Muuntaja support Sieppari -style interceptors too. See muuntaja.interceptor for details.

Interceptors can be used with Pedestal too, all but the exception-interceptor which conforms to the simplified exception handling model of Sieppari.

Configuration

Explicit Muuntaja instance with custom EDN decoder options:

(def m
  (m/create
    (assoc-in
      m/default-options
      [:formats "application/edn" :decoder-opts]
      {:readers {'INC inc}})))

(->> "{:value #INC 41}"
     (m/decode m "application/edn"))
; => {:value 42}

Explicit Muuntaja instance with custom date formatter:

(def m
  (m/create
    (assoc-in
      m/default-options
      [:formats "application/json" :encoder-opts]
      {:date-format "yyyy-MM-dd"})))

(->> {:value (java.util.Date.)}
     (m/encode m "application/json")
     slurp)
; => "{\"value\":\"2019-10-15\"}"

Returning a function to encode transit-json:

(def encode-transit-json
  (m/encoder m "application/transit+json"))

(slurp (encode-transit-json {:kikka 42}))
; => "[\"^ \",\"~:kikka\",42]"

Encoding format

By default, encode writes value into a java.io.ByteArrayInputStream. This can be changed with a :return option, accepting the following values:

value	description
`:input-stream`	encodes into `java.io.ByteArrayInputStream` (default)
`:bytes`	encodes into `byte[]`. Faster than Stream, enables NIO for servers supporting it
`:output-stream`	encodes lazily into `java.io.OutputStream` via a callback function

All return types satisfy the following Protocols & Interfaces:

ring.protocols.StreamableResponseBody, Ring 1.6.0+ will stream these for you
clojure.io.IOFactory, so you can slurp the response

`:input-stream`

(def m (m/create (assoc m/default-options :return :input-stream)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; #object[java.io.ByteArrayInputStream]

`:bytes`

(def m (m/create (assoc m/default-options :return :bytes)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; #object["[B" 0x31f5d734 "[B@31f5d734"]

`:output-stream`

(def m (m/create (assoc m/default-options :return :output-stream)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; <<StreamableResponse>>

Format-based return

(def m (m/create (assoc-in m/default-options [:formats "application/edn" :return] :output-stream)))

(->> {:kikka 42}
     (m/encode m "application/json"))
; #object[java.io.ByteArrayInputStream]

(->> {:kikka 42}
     (m/encode m "application/edn"))
; <<StreamableResponse>>

HTTP format negotiation

HTTP format negotiation is done using request headers for both request (content-type, including the charset) and response (accept and accept-charset). With the default options, a full match on the content-type is required, e.g. application/json. Adding a :matches regexp for formats enables more loose matching. See Configuration docs for more info.

Results of the negotiation are published into request & response under namespaced keys for introspection. These keys can also be set manually, overriding the content negotiation process.

Exceptions

When something bad happens, an typed exception is thrown. You should handle it elsewhere. Thrown exceptions have an ex-data with the following :type value (plus extra info to enable generating descriptive erros to clients):

:muuntaja/decode, input can't be decoded with the negotiated format & charset.
:muuntaja/request-charset-negotiation, request charset is illegal.
:muuntaja/response-charset-negotiation, could not negotiate a charset for the response.
:muuntaja/response-format-negotiation, could not negotiate a format for the response.

Server Spec

Request

:muuntaja/request, client-negotiated request format and charset as FormatAndCharset record. Will be used in the response pipeline.
:muuntaja/response, client-negotiated response format and charset as FormatAndCharset record. Will be used in the response pipeline.
:body-params contains the decoded body.

Response

:muuntaja/encode, if set to truthy value, the response body will be encoded regardles of the type (primitives!)
:muuntaja/content-type, handlers can use this to override the negotiated content-type for response encoding, e.g. setting it to application/edn will cause the response to be formatted in JSON.

Options

Default options

{:http {:extract-content-type extract-content-type-ring
        :extract-accept-charset extract-accept-charset-ring
        :extract-accept extract-accept-ring
        :decode-request-body? (constantly true)
        :encode-response-body? encode-collections}

 :allow-empty-input? true
 :return :input-stream

 :default-charset "utf-8"
 :charsets available-charsets

 :default-format "application/json"
 :formats {"application/json" json-format/json-format
           "application/edn" edn-format/edn-format
           "application/transit+json" transit-format/transit-json-format
           "application/transit+msgpack" transit-format/transit-msgpack-format}}

Profiling

YourKit supports open source projects with its full-featured Java Profiler. YourKit, LLC is the creator of YourKit Java Profiler and YourKit .NET Profiler, innovative and intelligent tools for profiling Java and .NET applications.

License

Picture

By Unknown. The drawing is signed "E. Ducretet", indicating that the apparatus was made by Eugene Ducretet, a prominent Paris scientific instrument manufacturer and radio researcher. The drawing was undoubtedly originally from the Ducretet instrument catalog. [Public domain], via Wikimedia Commons.

Original Code (ring-middleware-format)

This library

Distributed under the Eclipse Public License 2.0.

arohner / muuntaja