OpenRTI ------- An rti library with an rti1516, rti1516e as well as a RTI13 interface. The key features are: * Easy to use. * C++ interface. * (Java interface. No coding resources allocated yet ... hint) * Extensible. * In process RTI. * Server tree hierarchy. * Shortest path is in memory without copying data. * Only depends on a c++98 compiler/stl or newer. Installation ------------ It's a usual cmake project. Execute cmake from within a build directory of your choice, give this cmake the source directory, generate build files for your native build environment and compile the set of libraries/tests. Specifically under a UNIX like environment: cd <builddir> cmake -DCMAKE_INSTALL_PREFIX=<prefix> <srcdir> make make test make install Specifically under windows using cl: cd <builddir> cmake -DCMAKE_INSTALL_PREFIX=<prefix> <srcdir> cmake --build . --config Release ctest --build-config Release cmake --build . --config Release --target INSTALL Or depending on your c++ compile environment, you may need to take different steps. For details refer to www.cmake.org. Dependencies ------------ The 'dependency only on a c++ compiler' in the key features table means more specifically that: The rti libraries and all tool programs like the server process rtinode that are required to run a rti, the client rti libraries as well as the complete test suite depends on nothing but a working c++ compiler and a usually included stl implementation. This aims to ease the installation and should keep potential collisions with respect to run time libraries possible required by the calling application at a bare minimum. OpenRTI requires at least an c++98 implementation, but can be compiled also with a c++11 compiler and stl library. OpenRTI can make use of additional components like zlib for on the wire compression, that is used if it is available. OpenRTI's on the wire protocol negotiates if this is used or not. So, mixing different builds one with zlib and one without shall interact seamlessly. The python binding is obviously only available if python is available. That said, much more specifically, there is no dependency on boost as of today and there will not be any in the future. The same goes for any other dependency library that is not strictly needed. C++ language standard version ----------------------------- C++11/14 Should just work. Starting with cmake 3.1 you can enable use of an other standard version conveniently in the build system by setting the CMAKE_CXX_STANDARD cmake variable to an appropriate value. For example switching to c++11 can be achieved via adding the -DCMAKE_CXX_STANDARD=11 command line option to the initial cmake command. Consult the cmake documentation for more information on setting cmake variables. Platforms --------- Everywhere! Seriously, if it does not work yet on your platform, please provide patches to make it work. Development is done currently on Linux, Win32, MacOS and Solaris. HP-UX did work at some time in the past but is no longer tested. Linux is tested on current fedora variants and centos 4, 5 and 6. Windows is tested so far with 32 and 64 bits on XP with VS 2010/VS2012. Macos X is fully supported starting with 10.6. Starting with this version a workaround for a problem originating from the standardized cyclic link dependency is available. On earlier macos versions OpenRTI triggers a race in resolving dynamic symbols from the logical time libraries. This problems shows up in a lot of the tests for OpenRTI. For single threaded HLA applications or applications having only a single thread accessing the RTIambassador this problem should not show up even on macos versions prior to 10.6. On Solaris, OpenRTI is tested with CC='cc -mt' CXX='CC -mt' cmake ... On HP-UX, OpenRTI is tested with CC='cc -Aa -mt' CXX='aCC -AA -mt' cmake ... Supported RTI Features ---------------------- In general OpenRTI supports HLA1.3, rti1516, rti1516e interfaces. The implementation among them is on the same level - according to the features available on the appropriate level. The basic transport is currently tcp/ip based so everything is basically delivered reliable. Object Model Modules are supported for rti1516e. Update rate reduction is not implemented. * Federation Management: Connect/Disconnect is available. Create/Destroy including Modules is available. Join/Resign including Modules is available. List is available. Synchronization Points are available. Save/Restore is not available. Connection Lost is available (1). * Declaration Management Publish/Subscribe for Object Class Attributes is available. Publish/Subscribe for Interaction Classes is available. Start/Stop Registration for Object Class is called whenever published (2). Turn On/Off Interactions is called whenever published (2). * Object Management Reserve/Release (Multiple) Object Instance Name is available. Register/Discover/Delete/Remove Object Instance is available. Update/Reflect Attribute Values is avaiable. Send/Receive Interaction is available. Local Delete Object Instance is missing. Attributes In/out Of Scope is missing. Request/Provide Attribute value update is available. Turn Updates On/Off is missing. Request Change/Query/Report Attribute Transportation is missing. Request Change/Query/Report Interaction Transportation is missing. * Ownership Management Is missing. * Time Management Is available with the exception of message retraction. * Data Distribution Management Is missing. * Support Services Are available. * Management Object Model Is present with the object model but does not provide any data or methods. (1) The general behavior of OpenRTI on lost connections does not conform to the standard. But the cases where OpenRTI recognizes a lost connection in the sense of the standard it is reported by the Connection Lost service. (2) Rather than tracking subsriptions the service is only called once the current rti ambassador published itself. This helps for applications that rely on the callbacks being called but does not help for reducing the load on the application. Operation --------- OpenRTI can operate in several ways: The most important one is probably the client/server scenario: Have one server process and some federates in several processes on some machines in a network. Often it would be also good to exploit some locality in an RTI. To get this, an OpenRTI server can operate as slave servers under a parent server. That means that every server in that tree seed all the federations in that server tree but messages are only routed to servers that need these messages. So having a sub server on a local subnet, traffic just interesting for federates connected to that slave server originating from federates connected to that slave server will never be sent to the parent server. But OpenRTI also can work without any external server running. When all federates are living in one program - may be in different threads, messages are just exchanged in that program. While this is probably the fastest operation mode it is also the one that is the easiest to set up. Connecting to an rti -------------------- As of this writing, there are three different c++ bindings available and implemented. All of them have different ways to connect, to an rti create a federation execution and finally join a federation execution. With OpenRTI all of them share a common backend implementation including the means of communication. Below a general description followed by the way how the possible connections are encoded into the api calls. Protocols --------- To select between the different modes of operation above, OpenRTI knows some protocols. This is the basic way of the transport used to communicate. rti: This is the preferred tcp/ip protocol variant for networked communication. This is the default with a default hostname of localhost. thread: Just use the in memory communication between threads within a single application. Requires no external server setup and is extremely fast. rtic: Like rti above but compresses the data stream if available on both transport ends. Compression needs extra cpu cycles and introduces some latency spent in compression of the transfered messages. Nevertheless if your available bandwidth is limited, this can help getting the data over the line. This url is accepted even if compression is not available on one or both of the communication ends. In such a case the data is not compressed before it gets sent over the wire. http: (currently just an idea) This is a less efficient variant of the rti protocol for networked communication. The trick is that his kind of connection uses standard http headers and encapsulates the rti messages into that. It should be possible to bypass firewalls with that protocol. Not yet sure if this idea also helps for proxies. Note that this does not use the CONNECT proxy method but really uses native http messages. Which simplifies configuration a lot. pipe: Uses the rti binary protocol on a named pipe. Can be used for machine local communication in presence of tight packet filters. rtinode: (experimental, only available in the ambassador) A meta protocol to be able to run an rtinode just from inside of any ambassador. The protocol uses url query key value pairs to configure this rtinode like server. Query keys understood are: config=<config file name> to provide a configuration file for rtinode listen=<listen url> to provide a listen url for rtinode parent=<parent url> to provide a parent url for rtinode Note that you may need to escape the queries properly if they contain characters that make the url parsing ambiguous else. A simple example url setting up an rtinode on localhost on the default port reads "rtinode:///FederationExecutionName?listen=rti:///". The implementation is prepared to some degree to move protocols into a user provided shared library that could be loaded at runtime. Federation execution names and URLs ----------------------------------- For rti1516 it is easy to set the connect parameters in the string list. But the RTI13 layer does not have this. So, OpenRTI interprets federation execution names as urls: <protocol>://<address>/<path>/<name> Where the protocol overrides the protocol field from the string list. The address is the network address of the server. The path contains the path to the named pipe if protocol is pipe and the name is just the plain name of the federation execution. This method does clearly break the rti1516 standard as you can connect to the same federation execution by different federation execution names depending on your server setup. But anyway I think that this is a useful idea to address different servers without relying on environment variables. So, for the standard conformance question we will need some override somewhere in the future to disable interpretation of federation execution names as urls. Connecting to an rti with RTI13 ------------------------------- This oldest binding does not have specific connect calls or arguments to any factory creating an RTI::RTIambassador. By that the RTI13 binding accepts the above described urls in any of calls refering to a federation execution. For example ambassador.joinFederationExecution("rti://myserver:3324/JustMine") will try to connect to myserver on port 3324 with the rti protocol and join the Federation 'JustMine' there. Connecting to an rti with rti1516 --------------------------------- For rti1516 we have that string vector that is given to the ambassador when it is created. This vector can be used to configure the protocol that is used to communicate. The strings might contain key value pairs like <key>=<value>. Supported keys are: protocol=<p> 'thread' (default) use the smp, in memory connect. 'rti' use the tcp/ip binary protocol on some custom port 'pipe' use the binary protocol on a named pipe address=<a> The interpretation depends on the protocol: For 'thread' this is ignored. For 'rti' this contains the host:port pair. port can be omitted to use the appropriate default port. For 'pipe' this should contain the path to the named pipe. host=<a> hostname=<a> Like address, but only sets the host name or ip address part. port=<p> service=<p> Like address, but only sets the port number or service name part. timeout=<t> Sets initial packet timeouts for messages that happen synchronous. That is mostly for create, destroy, join and resign. url=<u> You can directly use the url scheme as described above and as it is used with rti1516e in the arguments to rti1516. Connecting to an rti with rti1516e ---------------------------------- The most recent rti1516e binding has a connect call which takes an url as described above to connect to an rti. The createFederationExecution and joinFederationExecution service finally takes the federation execution names as mandated by the standard. Server ------ The standalone server application is called 'rtinode'. The default operation of rtinode is to listen and respond on any ip adress at port 14321 for incomming rti connections. Accepted command line options are: -b Run in the background. -c <configuration file> Read the server configuraton from the given configuration file. -f <socket file> Start up a server only listening on the provided unix socket file name. This corresponds to a pipe:///<socket file>/<FederationName> url at ambassador connect time. The option is only available on unix like operating systems. -i <inet address> Start up a server only listening on the provided tcp/ip address. Valid adresses are address:port pairs like you find in a lot of ip based server configurations. This corresponds to a rti://address[:port]/<FederationName> url at ambassador connect time. Provided with an url used for connecting to a server node, the protocol/address/service is extracted from that and a listening server set up accordingly. To determine the tcp/ip address to bind the server, the address/service provided is split into a host/service pair separated by the last double colon that is found in the address and this pair is entered into the getaddrinfo libc call. The first working address is used to put up a listening socket for rtinode. That means that the exact address that is used to listen on depends on your systems behavior and name resolution setup. This is expecially true for the selection of ipv4/ipv6 and the resolution of names provided with the host or service part of the above argument. Nevertheless, a set of commonly working examples are: '*:1234' to listen on any address on port 1234. '[::]:14536' to listen on the any ipv6 address on port 14536. 'localhost' to restrict listening to the ip address that is configured for localhost. The result depends on what your system thinks 'localhost' should be. Since no port is given the default 14321 port is used. -p <url> Act as a child server by connecting to the parent server at the provided url. LogicalTime ----------- As required by rti1516e, OpenRTI knows about 64 bit integer and 64 bit float times. The provided logical time implementations should fulfill the table as found in paragraph 12.3 of the rti1516e standard document: // let Iz = 0, Ie = epsilon, Iz < I // any T denotes a value of type time, // any I denotes a value of type interval: // // T' = T + I then T' > T (*) // T' = T - I then T' < T // T' = T + Iz then T' == T // T' = T - Iz then T' == T // T1 != T2 then |T1 - T2| >= Ie // T1 == T2 then T1 - T2 == Iz // // I' = I + Iz then I' == I // I' = I - Iz then I' == I // I1 != I2 then |I1 - I2| >= Ie // I1 == I2 then I1 - I2 == Iz // I != Iz then I >= Ie // // T > T' then T' < T For integer valued logical times this properties are trivially valid. For float valued logical times this is reached by using the smallest representable double value > 0, which is the denormalized minimum as epsilon. The '+' operator used with a strictly positive and thus non zero time interval always results in a logical time that is strictly greater than the input time to this operation. This is property (*) in the above table. While the full set of properties may be needed for time management to work properly, the property (*) is critical and easily not valid for a naive implementation of float valued logical time. The tests/rti1516/time/logical-time test program can be used to test a logical time implementation for the required behavior. With included float valued logical time implementations, any attempt to compare or encode a NaN valued logical time or logical time interval will result in an invalid time exception. This is to avoid internal problems with NaN's, specifically the last property in the table cannot be fulfilled with NaN values but is needed for the use of logical times as key values in std::map which requires strong ordering on the key. Else the underlying implementation may behave undefined and with that OpenRTI may behave undefined also. Encoding -------- For all this binary communication, OpenRTI always sends an initial connect packet and a server always answers with the same type of packet. This packet starts with a 12 byte fixed header containing the projects name 'OpenRTI\0' and the 4 byte big endian total packet size. The payload of that packet is a string key/value map carrying some information about the client. This includes the client version number, the set of available encodings and the clients endianess. The string based approach should be extensible enough to build up a backward compatible communication setup. The server then selects the best encoding that matches the client and may be some server configured policy like compression for example. Then the server responds with such a string map containing the selected encoding. All following messages are exchanged in that selected encoding. In the best case, this leads to optimal encoding. In the worst case we have a controlled channel to refuse a connection. The client can then abort and return with an error message to the user. The implementation is prepared to some degree to move encodings into a user provided shared library that could be loaded at runtime. Locking: -------- OpenRTI tries to lock object as little as possible. The implementation assumes that each component lives in one thread. That means, there is no need for locking for almost all operations OpenRTI's components will do. The critical part is when messages are sent or received between threads. This always happens through some kind of a locked message queue. Once a message is sent to such a message queue, the sender should never touch that message again. Due to messages being reference counted, the last one switches the light off... There are two notable exceptions to this. 1. The thread rti server: Since this rti server does not have its own thread to live in, its execution is guarded by a mutex that serializes sending messages to this server. 2. The VariableLengthData: See next section. VariableLengthData Lifetime. ---------------------------- The VariableLengthData class is implemented using a reference counted copy/allocate on write scheme. This means that passing these values around has just the cost of reference counting. In the best case and with federates living in the same application, the receiving federate sees exactly the same payload memory than the sending federate used to write into. For that copy on write to work no locking is required. Server state: ------------- FIXME Publication and Subscription: ----------------------------- This is a brief overview how publication and subscription are propagated over the tree. Publication is a simple two state value that is tracked for each connect in a server node. When a server node or ambassador behind a connect changes this state it sends an appropriate message to the server node. The server node tracks this publication state for each connect. This server node is responsible to inform the other connects about its publication state. A server node should only inform once this state changes for each connect. When the server node receives a second publication through a second connect, no publication state should be changed for connects that already know about that. Publication state is immediately pushed to a new server node once it is registered. Subscription is tracked similar to publication. But subscription is only sent to those connects that publish. That means if we get some delayed publication, all current subscriptions need to be sent to that server node/ambassador. Object Instance Management: --------------------------- Object instance handles need to be globally managed to stay unique. Also Object instance handles have lifetimes beyond the timestamped delete instance message sent from the owner. Thus a federate that has an object handle in use in any sense keeps a reference of this object instance handle in its parent server node. Once all federates have dropped their references the handle is again free and can be reused. Together with every object instance handle the root server also allocates a unique object instance name. This holds also for the object instance name reservations, where the instance handle is already assigned once the name is reserved. Every server node tracks the child connects that reference an object instance handle. Once there are no old connects left that reference an object instance handle, the server node unreferences this handle at its parent. The federate needs to release all the references it owns explicitly even on destruction. This is to help the server node with the cleanup since the server nodes only know which connect has a reference to a handle not which federate. In contrast to that, if a connection dies, may be due to a crashed federate, the server node will just unreference the connect handle at all object instance handles it references. This way we can guarantee a consistent cleanup even on faulty federates.