DBussy is yet another Python binding for accessing D-Bus https://www.freedesktop.org/wiki/Software/dbus/. I know there is already dbus-python http://dbus.freedesktop.org/doc/dbus-python/, among others https://www.freedesktop.org/wiki/Software/DBusBindings/. So why do we need another one?
The main issue is one of event loops. Most of the existing bindings seem to be based around GLib. However, Python now has its own “asyncio” event-loop architecture https://docs.python.org/3/library/asyncio.html. This goes back to Python 3.4, but as of 3.5, you now have full-fledged coroutines (“async def” and “await”) as a language feature.
Every GUI toolkit already provides its own event loop; so why did the Python developers decide to add yet another one? The answer seems clear: to provide a language-standard API for event loops, and a reference implementation for this API. It should be possible to adapt other event loops to this same API, and then Python code written to work with asyncio becomes event-loop agnostic.
D-Bus is a high-level interprocess communication protocol. It also provides a standard daemon, that is included with the main Linux desktop environments, that implements a set of standard “buses”: a “system” bus that is created at system boot time, and a “session” bus that belongs to each user who logs into one of these desktop environments.
Processes can register their services on one of these buses--the system bus for systemwide access, or the session bus for per-user access--where other processes can find them by name and connect to them. Or they can accept connections on entirely separate networking sockets, without any dependency on the D-Bus daemon. libdbus, the reference implementation for the low-level D-Bus protocol, supports both modes of operation.
D-Bus is based around the concept of passing messages conforming to a standard, extensible format. Messages are of four types:
- a “method call”
- a “method return” (normal response to a method call)
- an “error” (abnormal response to a method call)
- a “signal” notification
A method-call message is how one process requests a service of another process via D-Bus; the usual response would be a method-return message in the other direction indicating the completion status of the service being performed; it is also possible to send method-call messages without expecting a reply. If there was something wrong with the method-call message (e.g. inappropriate parameters, lack of permissions), then the response would be an error message. One could also send a method-return with information indicating a failure to perform the requested service; presumably the choice between the types of response is that an error return indicates a condition that is not supposed to happen--a bug in the requesting program.
Signal messages are sent as notifications of interesting events pertaining to the current session (for the session bus) or the entire system (for the system bus). They are usually not sent to a specific destination, but can be picked up by all interested processes on the bus. There are no replies to signals; if the receiving process cannot or will not process a particular message, it simply ignores it.
Messages optionally include the following information:
- a destination “bus name” indicating the process that is to receive the message (this is not the name of the bus, but the name of a process on the bus)
- an “object path” which looks like a POSIX absolute file name (always beginning with a slash and never ending with a slash, except for the root object “/”); the meaning of this is up to the receiving process, but it is intended to indicate some object within the hierarchy exposed by the process
- an “interface name” which identifies the particular message protocol
- a “method name” which identifies the particular function to be performed within that interface.
Bus names and interface names look like domain names with the components reversed, so the top level is at the beginning. If you are familiar with package names in Java, they take the same form, and with the same intent: to reduce the chance of name conflicts.
D-Bus also includes an extensive, but not extensible, type system for encoding data in a message. This data represents arguments to the method call or signal, return results for a method return or the error name and message for an error. A message contains a sequence of 0, 1 or more items of such data, each of which can be of various types: “basic” types (e.g. integer, float, string) or “container” types (structs, arrays, dictionaries) which in turn contain more values, each of which in turn can be of a basic or (recursively) another container type. A “signature” is a string encoding the type of a value, or sequence of values; there is also a “variant” type, which means the type of the value is encoded dynamically with the value itself, separate from the signature.
The importance of type signatures is really up to the particular programs that are trying to communicate: some might insist on values exactly matching the expected type signature, whereas others might be more lenient. For example, while the D-Bus type system specifies different basic types for different sizes of integers of signed or unsigned varieties, most Python code will probably not care about the specific distinctions, and treat all these values as of type “int”.
D-Bus defines some standard interfaces which are meant to be understood by most if not all services.
One fundamental one is the “org.freedesktop.DBus.Introspectable” interface; this defines an “Introspect” method, that is expected to return an XML string that describes all the interfaces understood by the object identified by the object path, as well as listing all the available child objects that can be accessed by appending a slash and the child name to the parent object path, if any. Introspection is a very important part of D-Bus: it is what allows users to discover what services are available on their installations, and throw together ad-hoc scripts in Python or other high-level languages to make convenient use of such services, without having to write a lot of interfacing code.
Another commonly-supported interface is called “org.freedesktop.DBus.Properties”. This one defines the concept of properties, which are pieces of data notionally attached to object paths, and which might be readable, writable or both. This interface defines standard methods to get a property value for an object, set a new property value, or get all properties defined on an object. It also specifies a signal that can be sent by a server process as a general notification to all peers on the bus about changes to its property values.
DBussy allows you to take advantage of asyncio, but it doesn’t force you to use it. DBussy is meant to give you access to (nearly) all the functionality of the underlying libdbus library https://dbus.freedesktop.org/doc/api/html/index.html. libdbus is a very low-level, very general library, designed to be called from C code, that makes no assumptions about event loops at all. Consider the basic task in a client program of sending a D-Bus request message and waiting for a reply; or consider a server waiting for a message to come in. libdbus offers 3 different ways to handle this:
- poll repeatedly until the message appears
- block the thread until the message comes in
- specify a callback to be notified when the message is available
DBussy offers access to all these ways. But it also gives you the option of engaging the asyncio event loop. This means you can be doing other things in the loop, and when a message comes in, it can be passed automatically to a callback that you previously specified.
It also gives clients another way of sending a message and waiting for the reply: using a coroutine. For example
async def request_reply(connection) :
message = dbussy.Message.new_method_call(...)
... other setup of message args etc ...
reply = await connection.send_await_reply(message, timeout)
... process reply ...
#end request_reply
loop = asyncio.get_event_loop()
dbus_task = loop.create_task(request_reply(connection))
... maybe create other tasks to run while awaiting reply ...
loop.run_until_complete(dbus_task)
On the server side, you can correspondingly use coroutines to handle time-consuming requests without blocking the main loop. A message filter or object-path message callback can return, instead of DBUS.HANDLER_RESULT_HANDLED, a coroutine. The wrapper code will give libdbus a result of DBUS.HANDLER_RESULT_HANDLED on your behalf, after creating a task to execute the coroutine on the event loop. The coroutine can then go on to handle the actual processing of the request, and return the reply at some later stage.
The dbussy module also offers several more Pythonic facilities beyond those of the underlying libdbus, including a higher-level representation of type signatures as Type objects (and subclasses thereof), and an Introspection object hierarchy that can be easily converted to and from the standard D-Bus introspection XML representation.
Rather than directly manipulating D-Bus message objects, it is usually more convenient to have a representation where D-Bus object paths are directly mapped to Python objects, and D-Bus method and signal calls are similarly mapped to calls on methods of those Python objects. So on the client side, the local Python objects become “proxies” for the actual objects implemented by the remote server. And on the server side, the implementation of an interface can be wrapped up in an “interface class” with methods that are automatically invoked in response to incoming D-Bus requests.
Interface classes can also be used on the client side: in this situation, the method calls are just stubs used for type-checking outgoing requests, while the signal definitions can be real functions which are invoked in response to incoming signal messages. Conversely, on the server side, the signal definitions are stubs used for type-checking, while the method definitions (in the D-Bus sense) are real functions implementing those calls.
An interface class can also be both client-side and server-side in one, which means all the method definitions are real, none are stubs. So it can be used both for type-checking outgoing messages and handling incoming ones. For example, this is true of both the common standard interfaces (Introspectable and Properties), since most if not all peers are expected to support them.
(Signal definitions are a special case: even in a client-and-server-side interface, they can be marked as stubs--as in the standard PropertyHandler interface. This allows you to register such an interface for introspection purposes, without having to accept its handling of any signals.)
Both kinds of interface representations are provided by the “ravel” module--interface classes on the client or server side, and proxy interfaces on the client side. Ravel also offers different ways to construct a proxy interface: you can define it yourself, or you can have Ravel construct it automatically for you by introspecting the server-side object.
Either way, you start by creating a ravel.Connection object, which is a wrapper around a lower-level dbussy.Connection object. You can get one for the session or system bus by calling ravel.session_bus() and ravel.system_bus() respectively, or you can use a ravel.Server object (wrapping around the corresponding dbussy.Server) to accept connections on your own network address, separate from the D-Bus daemon.
Proxy interfaces can be easily constructed in different ways. One way is to start with a proxy for a bus peer with a particular name. You get one of these with an expression that treats the connection as though it were a mapping:
peer = conn[«bus_name»]
Then you do another lookup on this mapping to get a reference to a particular object path at that peer:
obj = peer[«object_path»]
Now, you can get a proxy for a desired interface thus:
iface = obj.get_interface(«interface_name»)
which causes automatic introspection of that object path on the peer to obtain all the necessary type information for that interface (if it is not one of the standard interfaces). So calling a Python method on this object
results = iface.«method»(«args»)
translates automatically to the corresponding D-Bus method call to that object and interface on the remote server, with full type checking done on both arguments and results.
Note that the method result is always a list.
D-Bus properties are automatically mapped to Python properties, so you can access their values and assign new ones in the usual Python way. For example, adding 1 to a numeric property (written out the long way to demonstrate property access on both the LHS and RHS of the assignment):
iface = conn[«bus_name»][«path»] \
.get_interface(«interface_name»)
iface.«prop» = iface.«prop» + 1
The above are blocking calls, which means the current thread is blocked while waiting for the reply to the method call. If you want to do things in a more event-loop-friendly fashion, then use get_async_interface instead of get_interface, which returns a coroutine object that evaluates to an asynchronous version of the proxy object when it finally completes. Method calls and property accesses on this are automatically also coroutine calls, so you can use them in await-constructs in your coroutines, or create asyncio tasks to run them etc.
Here is an example use of the above calls, which pops up a GUI notification displaying a short message for 5 seconds. Because the introspection of this interface supplies names for the arguments, it is possible to pass them to the method call by keyword:
ravel.session_bus()["org.freedesktop.Notifications"]["/org/freedesktop/Notifications"] \
.get_interface("org.freedesktop.Notifications") \
.Notify \
(
app_name = "test",
replaces_id = 0,
app_icon = "dialog-information",
summary = "Hello World!",
body = "DBussy works!",
actions = [],
hints = {},
timeout = 5000,
)
(But note that the argument names might differ, depending on your Linux distro version. If you get errors saying certain argument names are not understood, try it without the argument names. Or do your own introspection of the interface, to decide what argument names should be used.)
The above access to proxy interfaces could be described as “bus name-path-interface”, after the order in which the components are specified. Proxies can also be obtained in “bus name-interface-path” order. This can be convenient for obtaining a proxy interface object that can then be used to make calls on multiple objects.
In this method, an initial root proxy is obtained thus:
iface_root = conn[«bus_name»].get_interface
(
path = «initial_path»,
interface = «interface_name»,
)
Note you need to specify an object path for this initial introspection; it is probably best to use the shortest (highest-level) path that supports that interface. Depending on the peer, the root path “/” might work.
From this, you get the proxy for the interface on an actual object by using the object path as a lookup key, e.g.
iface = iface_root[«path»]
From here, you can invoke the method calls and access properties in the same way as before, e.g.
iface.«method»(«args»)
... iface.«prop» ...
As mentioned, both property and method access can be done asynchronously on an event loop. Asynchronous reading of a property is easy enough to express:
val = await obj.«prop»
But how do you do asynchronous writing of the property? The obvious construct
await obj.«prop» = newval
produces a syntax error: Python doesn’t (at least as of 3.6!) allow “await” on the left-hand side of an assignment. Instead, you write it as though it were a blocking call:
obj.«prop» = newval
but because the interface is defined as asynchronous, this causes a task to be queued on the event loop to oversee the completion of the set-property call, and your code can continue execution before this task completes.
The main consequence of this is that any error exception will be raised asynchronously. But if you don’t like the idea that execution will be deferred, you can await the completion of all such pending property-setting calls with the following call on the root proxy:
await iface.set_prop_flush()
This means you can batch up a whole series of property-setting calls on any number of objects on the same interface and bus name, then wait for them all to complete with a single flush call.
The more structured high-level interface offered by Ravel is built around the concept of an interface class, which is a Python class that represents a D-Bus interface, either as an actual implementation or as a “proxy” for making calls to another bus peer. You can then register instances of this class on a bus connection at selected points in your object path hierarchy, to handle either only specific objects at those paths or as a fallback to also deal with objects at points below those, that do not have their own instance of this class registered.
An interface class is identified by applying the @ravel.interface() decorator to the class definition, specifying the kind of interface (for use client-side, server-side or both), and the interface name, e.g.
@ravel.interface(ravel.INTERFACE.SERVER, name = "com.example.my_interface")
class MyClass :
...
#end MyClass
The meanings of the first, “kind”, argument to @ravel.interface are as follows:
- INTERFACE.SERVER -- you are the server implementing the method calls defined by this interface. However, the signal definitions are just “stubs” used for type-checking when you send those signals over the bus. This interface definition can also be introspected to inform users about the facilities provided by the interface.
- INTERFACE.CLIENT -- you are a client wanting to communicate with a server that implements this interface. The method calls are just stubs used for type-checking when you send those calls over the bus. The signal definitions can be your actual functions that you want to be invoked when those signals are received, or they can also be stubs.
- INTERFACE.CLIENT_AND_SERVER -- both of the above; you implement the methods, and maybe the signals as well, and you can also use their definitions to send corresponding method and signal calls to peers that implement the same interface. The standard interfaces (Peer, Introspectable, Properties) are defined in this way.
Within such a class, Python methods that are to handle D-Bus method calls are identified with the @ravel.method() decorator, e.g.:
@ravel.method \
(
name = ...,
in_signature = ...,
out_signature = ...,
args_keyword = ...,
arg_keys = ...,
arg_attrs = ...,
result_keyword = ...,
result_keys = ...,
result_attrs = ...,
connection_keyword = ...,
message_keyword = ...,
path_keyword = ...,
bus_keyword = ...,
set_result_keyword = ...,
...
)
def my_method(...) :
...
#end my_method
As you can see, there are a large number of options for implementing such a method. It can also be defined as a coroutine with async def if you have an event loop attached, and Ravel will automatically queue the task for execution and await any returned result. Partial summary of arguments:
- name -- the D-Bus method name. If omitted, defaults to the Python function name.
- in_signature -- the D-Bus signature specifying the arguments (zero or more) to the method.
- out_signature -- the D-Bus signature specifying the results (zero or more) the method will return.
- args_keyword -- the name of an argument to the Python function that will be set to the arguments from the message method call. The arguments will be passed as a list, or a dict, or an attributed class, depending the specification of arg_keys and arg_attrs (see below).
- path_keyword -- if specified, then the object path field from the incoming method call will be passed to the Python function as the value of the argument with this name.
- message_keyword -- if specified, then the dbussy.Message object for the incoming method call will be passed to the Python function as the value of the argument with this name.
- connection_keyword -- if specified, then the dbussy.Connection object will be passed to the Python function as the value of the argument with this name.
- bus_keyword -- if specified, then the ravel.Connection object will be passed to the Python function as the value of the argument with this name.
- set_result_keyword -- if specified, then a function of a single argument will be passed to the Python function as the value of the argument with this name; the argument passed by calling this function becomes the method result.
Passing arguments: the argument with the name given by args_keyword will hold the extracted arguments from the method call message. If neither of arg_keys or arg_attrs is specified, then the arguments are passed as a list. If arg_keys is specified, then it must be a sequence of names that must match the number of types specified by the in_signature; in this case, the args will be passed as a dict with the keys given in arg_keys associated in order with the argument values.
If arg_attrs is specified instead of arg_keys, then it must be a sequence of names that must match the number of types specified by the in_signature; a mutable attributed class object is created by calling ravel.def_attr_class, with the attribute names taken from arg_keys assigned in order to the argument values.
Returning results: the function can return the result values to be inserted into the method-return message as the function result, by assigning to elements of a mutable result argument (passed as the argument named by result_keyword, or by calling the set_result function that was passed via the set_result_keyword (above).
If neither result_keys nor result_attrs is specified, then the result is expected to be a sequence of values matching the out_signature. If it is returned as the function result, then it can be a tuple or list; but if result_keyword is specified, then the value of this is a list, and the values in the sequence must be assigned to the elements of this list in-place.
If result_keys is specified, then the result is a dict mapping the names from result_keys to the values of the result sequence in order. If result_attrs is specified, then the result is a mutable attributed class object created by calling ravel.def_attr_class, mapping the names from result_attrs to the values of the result sequence in order. If result_keyword is not specified, then the result object is expected to be returned as the function result; otherwise, it is passed as the value of the argument named by result_keyword, and the handler is supposed to update its elements in-place.
Signal definitions look similar, except they return no results:
@ravel.signal \
(
name = ...,
in_signature = ...,
args_keyword = ...,
arg_keys = ...,
arg_attrs = ...,
connection_keyword = ...,
message_keyword = ...,
path_keyword = ...,
bus_keyword = ...,
stub = ...,
...
)
def my_signal(...) :
...
#end my_signal
Also note the “stub” argument--this has meaning on a client-side interface to indicate that the interface class does not implement the listener for the signal, but that it is registered separately with a listen_signal call. This is used for the PropertiesChanged signal in ravel.PropertyHandler (the standard handler for the DBUS.INTERFACE_PROPERTIES interface), so that you do not have to replace the class just to install your own listeners for this signal.
Properties are defined by implementing getter and/or setter methods, identified by @propgetter() and @propsetter() decorators respectively:
@ravel.propgetter \
(
name = ...,
type = ...,
name_keyword = ...,
connection_keyword = ...,
message_keyword = ...,
path_keyword = ...,
bus_keyword = ...,
change_notification = ...
)
def my_propgetter(...) :
...
return \
«value»
#end my_propgetter
@ravel.propsetter \
(
name = ...,
type = ...,
name_keyword = ...,
type_keyword = ...,
value_keyword = ...,
connection_keyword = ...,
message_keyword = ...,
path_keyword = ...,
bus_keyword = ...
) :
def my_propsetter(...) :
...
#end my_propsetter
Note the following arguments:
- type -- the type signature for permitted property values.
- change_notification -- one of the dbussy.PROP_CHANGE_NOTIFICATION values indicating whether (and what kind of) signals should be generated for changes to this property value. This is specified on the @propgetter(), because there is no point notifying about write-only properties.
- type_keyword -- for passing the actual type of the new property value to the setter.
- value_keyword -- for passing the new property value to the setter.
Getters and setters can be coroutines.
With Ravel’s interface classes, it is possible to attach your own user data items to arbitrary points in the object path tree. To obtain the user data dictionary for a given object path, do either
user_data = bus.user_data["/com/example/myapp"]
or
user_data = bus.user_data["com", "example", "myapp"]
The result is a dictionary into which you can insert whatever key-value pairs you like, e.g.:
user_data["com.example.myapp.attribs"] = MyObj(...)
Ravel provides predefined interface classes for the org.freedesktop.DBus.Peer, org.freedesktop.DBus.Introspectable and org.freedesktop.DBus.Properties interfaces, and these are automatically registered on Connection instances. The Peer interface is just a stub, since the actual implementation is hard-coded into libdbus itself; it is there to provide automatic introspection of this interface.
The ravel.IntrospectionHandler class defines the standard Introspectable interface, and provides automatic introspection of all interfaces registered with a ravel.Connection (including itself and the other standard interfaces). It extracts the information specified to the class, method, signal and property-handler decorators, and generates the appropriate XML form for returning to D-Bus queries.
The ravel.PropertyHandler class defines the standard Properties interface, and automatically dispatches to @propgetter() and @propsetter() methods as defined in your registered interface classes.
Sample code illustrating how to use DBussy/Ravel is available in my dbussy_examples repo on GitLab https://gitlab.com/ldo/dbussy_examples and GitHub https://github.com/ldo/dbussy_examples.
The name is a pun on “dbus” and the name of French Impressionist composer Claude Debussy. The most natural way to pronounce it would be the same as his name. At least, that’s my story, and I’m sticking to it.
Lawrence D'Oliveiro ldo@geek-central.gen.nz 2017 May 23