lesteve / emacs-zmq

Emacs bindings to ØMQ

Bindings to zmq in Emacs.

Installation

Dependencies

libzmq

https://github.com/zeromq/libzmq

Building

Run make in the top level directory to build the zmq module. Alternatively when running (require 'zmq) and the module is not built already, you will be asked to build it.

Note, automake must be available on your system in order to build the module.

By default a local copy of libzmq is built, if you would like to link your own version of libzmq set the environment variables ZMQ_CFLAGS and ZMQ_LIBS to an appropriate value before running make. Currently the --enable-drafts=yes configure flag is required.

The default version of libzmq built is 4.3.1 and can be changed by specifying the environment variable ZMQ_VERSION, e.g. to something like

ZMQ_VERSION=4.3.0

Windows

To be able to use this package on Windows, your Emacs needs to be compiled with module support. Currently, the official versions of Emacs appear not to have this option enabled by default so you will have to build an Emacs with module support by following the instructions here, which show a way to do this using the MinGW tools. You will need to pass the --with-modules option to the configure script when building. See the instructions below on how to install the MinGW tools.

Download the pre-built libraries

You can download a tar archive containing the pre-built Windows dll files necessary to use this package. Both an emacs-zmq.dll library and a v4.3.1 libzmq.dll are provided.

The archive can be downloaded from the releases page on the Github project. After downloading, extract the archive contents into the same directory as this project then start Emacs normally and (require 'zmq) should work as expected.

Build using MinGW

It is possible to use the included build chain on Windows using the MSYS2 MinGW tools.

Install the 64-bit toolchain inside the MSYS2 shell via

pacman -S base-devel
pacman -S git
pacman -S mingw-w64-x86_64-gcc

Note: If you are using the official Git for Windows instead of MSYS2 Git, make sure to set

git config --global core.autocrlf false

during the build to avoid EOL issues and set it back to true (the default on Windows) when you are done building.

Start the build from an MSYS2 MinGW 64-bit shell via make.

Testing

Run make test in the top level directory.

Contexts

To create a context:

(zmq-context)

Normally only a single context object for the current Emacs session is necessary so the usual way to get the context for the current Emacs session is to call zmq-current-context which will create a context for the session only if one has not been created already. See Context/socket/poller lifetime management.

Below is a table mapping the C API functions to their Emacs equivalent.

C	emacs-lisp
zmq_ctx_new	zmq-context
zmq_ctx_set	zmq-context-set
zmq_ctx_get	zmq-context-get
zmq_ctx_term	zmq-context-terminate
zmq_ctx_shutdown	zmq-context-shutdown

Sockets

To create a socket:

(zmq-socket (zmq-current-context) zmq-PUB)

To bind a socket:

(zmq-bind sock "tcp://127.0.0.1:5555")

To receive a message without blocking:

(let (msg)
  (while (null (condition-case err
                   (setq msg (zmq-recv sock zmq-NOBLOCK))
                (zmq-EAGAIN nil)))
    (sleep-for 1)))

Below is a table mapping the C API functions to their Emacs equivalent.

C	emacs-lisp
zmq_socket	zmq-socket
zmq_send	zmq-send
zmq_recv	zmq-recv
zmq_bind	zmq-bind
zmq_unbind	zmq-unbind
zmq_connect	zmq-connect
zmq_disconnect	zmq-disconnect
zmq_join	zmq-join
zmq_leave	zmq-leave
zmq_close	zmq-close
zmq_setsockopt	zmq-socket-set
zmq_getsockopt	zmq-socket-get

In addition to the above, there are also some convenience functions for working with sockets. Currently this is only the function zmq-bind-to-random-port which takes a socket and an address and binds the socket to a random port on the address:

(zmq-bind-to-random-port sock "tcp://127.0.0.1") ; returns port number

Messages

To create a new message object use zmq-message

(zmq-message)

The above creates and initializes an empty message. You can also pass a string or a vector of bytes to zmq-message to initialize the message with some data

(zmq-message "[mα, mβ] = iℏmγ")
;; Initialize a message with a vector of bytes
(zmq-message [0 10 100 29])

Below is a table mapping the C API functions to their Emacs equivalent.

C	emacs-lisp
zmq_msg_init	zmq-message
zmq_msg_init_data	zmq-message
zmq_msg_recv	zmq-message-recv
zmq_msg_send	zmq-message-send
zmq_msg_move	zmq-message-move
zmq_msg_copy	zmq-message-copy
zmq_msg_close	zmq-message-close
zmq_msg_data	zmq-message-data
zmq_msg_size	zmq-message-size
zmq_msg_more	zmq-message-more-p
zmq_msg_set	zmq-message-set
zmq_msg_get	zmq-message-get
zmq_msg_gets	zmq-message-property
zmq_msg_routing_id	zmq-message-routing-id
zmq_msg_set_routing_id	zmq-message-set-routing-id
zmq_msg_group	zmq-message-group
zmq_msg_set_group	zmq-message-set-group

Multi-part messages

To send a multi-part message:

(zmq-send-multipart sock '("part1" "part2" "part3"))

To receive a multi-part message:

(zmq-recv-multipart sock)

zmq-recv-multipart returns a list containing the parts of the message and always returns a list, even for a message containing a single part.

Polling

Currently, polling requires that libzmq be built with the draft API to expose the zmq_poller interface. Below is an example of how you may poll a socket.

(catch 'recvd
  (let ((poller (zmq-poller))
        (timeout 1000))
    (zmq-poller-add poller sock (list zmq-POLLIN zmq-POLLOUT))
    (while t
      ;; `zmq-poller-wait-all' returns an alist of elements (sock . events)
      (let* ((socks-events (zmq-poller-wait-all poller 1 timeout))
             (events (cdr (zmq-assoc sock socks-events))))
        (when (and events (member zmq-POLLIN events))
          (throw 'recvd (zmq-recv sock)))))))

Below is a table mapping the C API functions to their Emacs equivalent.

C	emacs-lisp
zmq_poller_new	zmq-poller
zmq_poller_destroy	zmq-poller-destroy
zmq_poller_add	zmq-poller-add
zmq_poller_add_fd	zmq-poller-add
zmq_poller_modify	zmq-poller-modify
zmq_poller_modify_fd	zmq-poller-modify
zmq_poller_remove	zmq-poller-remove
zmq_poller_remove_fd	zmq-poller-remove
zmq_poller_wait	zmq-poller-wait
zmq_poller_wait_all	zmq-poller-wait-all

Errors

All errors generated by the underlying C API are converted into calls to signal in Emacs. So to handle errors, wrap your calls to zmq functions in a condition-case like so

(setq poll-events
      (while (null (condition-case nil
                       (zmq-poller-wait poller 1)
                     (zmq-EAGAIN nil)))
        (sleep-for 1)))

The error symbols used are identical to the C error codes except with the prefix zmq-. Only the more common errors are defined as error symbols that can be caught with condition-case, below is the current list of errors that have error symbols defined:

EINVAL

EPROTONOSUPPORT

ENOCOMPATPROTO

EADDRINUSE

EADDRNOTAVAIL

ENODEV

ETERM

ENOTSOCK

EMTHREAD

EFAULT

EINTR

ENOTSUP

ENOENT

ENOMEM

EAGAIN

EFSM

EHOSTUNREACH

EMFILE

Any other error will signal a zmq-ERROR with an error message obtained from zmq_strerror.

Comparing ZMQ objects

There are also predicate and comparison functions available for working with ZMQ objects:

zmq-poller-p

zmq-socket-p

zmq-context-p

zmq-message-p

zmq-equal

zmq-assoc

zmq-equal and zmq-assoc work just like equal and assoc respectively, but can also compare ZMQ objects.

Getting/setting options

To set an option for a zmq-context, zmq-socket, or zmq-message call:

(zmq-context-set ctx zmq-BLOCKY nil)
(zmq-socket-set sock zmq-IPV6 t)
(zmq-message-set msg zmq-MORE t)

To get an option:

(zmq-context-get ctx zmq-BLOCKY)
(zmq-socket-get sock zmq-IPV6)
(zmq-message-get msg zmq-MORE)

Or the convenience functions zmq-set-option and zmq-get-option can be used which will call one of the functions above based on the type of the first argument:

(zmq-set-option ctx zmq-BLOCKY nil)
(zmq-set-option sock zmq-IPV6 t)

(zmq-get-option ctx zmq-BLOCKY)
(zmq-get-option sock zmq-IPV6)

To access a zmq-message meta-data property use zmq-message-property:

(zmq-message-property msg :identity)

The available metadata properties can be found in zmq-message-properties.

Boolean options

Integer options which are interpreted as boolean in libzmq are interpreted in Emacs as boolean. For example, the socket option zmq-IPV6 which enables IPV6 connections for the socket is an integer option interpreted as a boolean value in the C API. In Emacs this option is a boolean. So to enable IPV6 connections you would do

(zmq-socket-set sock zmq-IPV6 t)

and to disable them

(zmq-socket-set sock zmq-IPV6 nil)

Similarly for all other socket, message, or context options which are interpreted as boolean by the C API.

Context/socket/poller lifetime management

The underlying Emacs module takes care of freeing the resources used by a ZMQ object during garbage collection. As a special case if a socket gets garbage collected, the zmq-LINGER property will be set to 0 for the socket (http://zguide.zeromq.org/page:all#Making-a-Clean-Exit). You probably still want to call the appropriate destructor function once your done using an object though.

Asynchronous subprocess

There is also the function zmq-start-process which creates an Emacs subprocess that can then be used for all your zmq processing needs. You pass zmq-start-process a function to run in the subprocess:

(zmq-start-process
 `(lambda ()
    (let* ((ctx (zmq-current-context))
           (sock (zmq-socket ctx zmq-SUB)))
      BODY)))

Or if you supply a function with a single argument to zmq-start-process, a context will be created for you and passed as the argument to the function:

(zmq-start-process
 `(lambda (ctx)
    (let ((sock (zmq-socket ctx zmq-SUB)))
      BODY)))

For one-way communication between the parent Emacs process and the subprocess created with zmq-start-process you can use zmq-subprocess-send in the parent and zmq-subprocess-read in the child process. zmq-subprocess-send takes a process object as its first argument and an arbitrary s-expression as its second argument. The s-expression will be encoded and sent to the subprocess. Using zmq-subprocess-read in the subprocess takes care of decoding and returns the s-expression sent from the parent process.

(let ((proc (zmq-start-process
             `(lambda (ctx)
                (let ((poller (zmq-poller)))
                  ;; Poll for input on STDIN, i.e. input from the parent emacs
                  ;; process
                  (zmq-poller-add poller 0 zmq-POLLIN)
                  (catch 'exit
                    (while t
                      (when (zmq-poller-wait poller 100)
                        (let ((sexp (zmq-subprocess-read)))
                          (zmq-prin1 sexp)
                          (throw 'exit t)))))))
             ;; A filter function which prints out messages sent by the
             ;; subprocess.
             :filter (lambda (sexp)
                       (message "echo %s" sexp)))))
  ;; Let the process start
  (sleep-for 0.2)
  (zmq-subprocess-send proc (list 'send "topic1")))

To read data from a subprocess, a filter function can be passed to zmq-start-process. This filter function is similar to a normal process filter function but only takes a single argument which will be list read from the subprocess output. This means that the parent process ignores any output generated by the subprocess that isn’t a list.