AREG SDK

Project Status

Introduction

AREG (Automated Real-time Event Grid) is a communication engine that enables data transmission in IoT fog- and mist-network. It is lightweight, automated, featured, and works on different platforms. AREG uses an interface-centric approach to create a grid of services that allows multiple devices and software nodes to operate like distributed servers and clients. By automating real-time data transmission, and using both Client-Server and Publish-Subscribe models, AREG ensures reliable and efficient communication between connected software nodes.

Motivation
Interface-centricity
More than embedded
Composition
Clone sources
Quick build
Integration
Pipeline
Use cases and benefits
Examples
License
Call to action
Thank you all!

Motivation

Traditionally, devices act as connected clients to stream data to the cloud or fog servers for further processing.

As data is generated and collected at the edge of the network (mist network), it is logical to redefine the role of connected Things and enable network-accessible services (Public Services) on them, thereby extending the Cloud to the extreme edge. This approach serves as a strong foundation for implementing robust solutions such as:

Increase data privacy, which is an important factor for sensitive data.
Decrease data streaming, which is a fundamental condition to optimize network communication.
Autonomous, intelligent and self-aware devices with services directly in the environment of data origin.

[ ↑ Back to top ↑ ]

Interface-centricity

ORPC, or Object Remote Procedure Call, is a remote procedure call concept that targets an interface on an object. It enables the construction of a service mesh (or service grid), where applications offer reusable services, and programmable client objects request method execution of programmable server objects without needing to know their location in the network.

The ORPC concept is interface-centric, similar to object-oriented programming, and supports transparent handling of multiple instances of the same object. There are no protocol restrictions, although the bi-directional communication is necessary to send messages to all connected nodes. The programmable server objects are called Service Providers, and the programmable clients are called Service Consumers. This approach enables AREG to combine the features of action-centric (Client-Server / Request-Reply) and data-centric (Publish-Subscribe / PubSub) models.

[ ↑ Back to top ↑ ]

More than embedded

The interface-centric nature of AREG engine provides a distributed solution for embedded application. AREG is designed to provide a homogeneous solution for multithreading, multiprocessing, and internet communications through categorized services (Local, Public, and Internet). These services are software components with predefined interfaces, which methods are invoked remotely.

💡 Currently AREG engine supports Local (multithreading) and Public (multiprocessing) service categories.

AREG forms a fault-tolerant system that automatically discovers and automates communications between services, allowing developers to focus on application logic development. The system ensures:

The crash of one application does not affect the entire system.
The clients automatically receive service availability notifications.
The requests, responses, and notifications are invoked in their own thread context.

[ ↑ Back to top ↑ ]

Composition

At present, the Automated Real-time Event Grid software development kit (AREG SDK) comprises the following primary modules:

Multicast router (mcrouter): a console application or OS-managed service that facilitates the message routing.
The AREG engine: a framework library that enables communication between software components.
Code generator: a tool to generate service Provider and Consumer objects from interface documents.

Furthermore, the AREG SDK includes a variety of examples, which illustrate the robust characteristics and functionality of the AREG communication engine. Additionally, there are various user interface (UI) tools currently in the development stage that aim to simplify and expedite the process of designing services and applications, as well as analyzing and interacting with applications on remote machines.

[ ↑ Back to top ↑ ]

Clone sources

To obtain the AREG SDK source codes and the dependent modules, run the git-command in your projects folder:

git clone https://github.com/aregtech/areg-sdk.git

[ ↑ Back to top ↑ ]

Quick build

The source codes of the AREG SDK are implemented using C++17 and can be compiled on multiple platforms, CPUs, and compilers.

Supported compilers: GCC, Clang, Cygwin GCC, and MSVC.
Supported platforms: Windows, Linux, and macOS.
Supported CPU: x86, x86_64, arm, and aarch64.

To compile the sources, the following tools can be used:

Tool	Solution	Platforms	API	Quick actions to compile
`cmake`	`CMakeLists.txt`	Linux, Cygwin, Windows	POSIX, Win32	- Build with cmake. - Build in VSCode. - Build in MSVS.
`msbuild`	`areg-sdk.sln`	Windows	Win32	- Build with msbuild. - Open and build in MSVS.

After compilation, the binaries are stored in the <areg-sdk>/product/build/<family>-<compiler>/<platform>-<bitness>-<cpu>-<build type>/bin folder. For example, if the code is compiled with GNU GCC on a 64-bit Linux system with an x86_64 CPU and in release mode, the binaries will be in the ./product/build/gnu-gcc/linux-64-x86_64-release/bin subfolder of areg-sdk.

To obtain comprehensive information regarding builds and options for the AREG SDK, we recommend referring to the Software build page on the AREG SDK Wiki. This document provides a brief outline of the compilation process using default options.

💡 Other POSIX-compliant operating systems and compilers have not yet been tested. List of supported POSIX APIs.

Build with `cmake`

Click to show / hide Build with cmake.

To build the AREG source codes using CMake, ensure that you have cloned the sources properly as described earlier. Open the Terminal in the areg-sdk directory and follow these steps:

Initialize the cache and build configuration with default options (g++ compiler, release build, build examples and unit tests):

cmake -B ./build

Compile the sources, justify parallel jobs with -j option:

cmake --build ./build -j8

💡 By default, the source codes of AREG SDK are compiled with cmake tool without examples and unit tests. This was done to achieve minimal required build using cmake. To include examples and unit tests, the developers need to pass additional compiler parameters when configure as shown in this example:

cmake -B ./build -DAREG_BUILD_EXAMPLES:BOOL=ON -DAREG_BUILD_TESTS:BOOL=ON
cmake --build ./build -j8

💡 For instructions on how to customize builds with cmake tool, kindly refer to the Build with CMake Wiki page.

Build with `msbuild`

Click to show / hide Build with msbuild.

To build the AREG and examples using MSBuild, ensure that you have cloned the sources properly as described earlier. Open the Command Prompt in the areg-sdk directory and compile sources with default options (MSVC compiler, release build, build examples and unit tests):

MSBuild .

💡 For instructions on how to customize builds with MSBuild tool, kindly refer to the Build with MSBuild Wiki page.

Build with WSL

Click to show / hide Build with WSL.

The Windows Subsystem for Linux (WSL) allows developers to use Linux applications directly on Windows machines. This means that developers may clone, compile and use AREG engine in a popular Linux distributions. To install and update WSL on your Windows 10 machine, clone and compile AREG SDK sources, and run examples, please follow the step-by-step instructions outlined in the Build with Windows Subsystem for Linux (WSL) Wiki page.

Build with IDEs

Click to show / hide Build with IDE.

💡 This chapter focuses on building AREG and examples using Microsoft Visual Studio and Visual Studio Code. The other IDEs are currently not in the focus.

To build the AREG and examples using Microsoft Visual Studio or Visual Studio Code, ensure that you have cloned the sources properly as described earlier and follow the instructions.

Build with Microsoft Visual Studio

Open areg-sdk.sln in Microsoft Visual Studio.
Build the solution with MSVC.

Build with Visual Studio Code

Open the areg-sdk folder in Visual Studio Code.
In the Explorer panel, select CMakeLists.txt.
Right-click on the file, select Configure All Projects from the context menu and wait for completion.
Right-click on the file again and select Build All Projects to build the sources with the default options.

[ ↑ Back to top ↑ ]

Integration

The AREG SDK provides developers with the flexibility to build their own projects using the AREG Framework or integrate the framework into existing projects, based on their specific requirements. For detailed information, please refer to the Setup a project Wiki page of the AREG SDK. Here is a brief description to get started with your own project.

Start a Project

Click to show / hide Start a Project.

To start a project using the AREG Framework, follow these steps:

Access the areg-sdk-demo repository, which serves as a practical example for creating projects with the AREG Framework. You can use these repository as a reference or template to start your own projects or integrate the AREG Framework into an existing projects.
In the areg-sdk-demo repository, the thirdparty sub-directory contains a submodule named areg-sdk, which represents the AREG Framework.
The areg-sdk-demo repository includes several sample demo projects built on the AREG Framework. Use these projects as a reference or template to start own projects.
Configure the builds to match your project's requirements. Refer to the Step 3: Configure AREG Framework chapter in the Setup a project Wiki page of the AREG SDK for detailed customization instructions. Follow the specific instructions based on your chosen build tool:
- If using the CMake tool, create and configure the CMakeLists.txt file. Adjust variables and set values as needed.
- If using Microsoft Visual Studio or MSBuild, create and configure the msvc_setup.props property file in the same location as the Solution file. Adjust property values as needed.
Ensure that the variables and values are set according to your project's requirements.

By following these steps, you can effectively initiate your project using the AREG Framework.

Create Services

Click to show / hide Create Services.

To create your own service, we recommend reviewing the step-by-step example and guidance provided in the Hello Service! document, which demonstrates various scenarios for creating local and public service providers and service consumers. Additionally, refer to the file structure guidelines presented in the DEVELOP.md document.

Please note that a GUI-based service interface design tool is currently under development as an open-source project and will be included in the repository in the future.

To begin development:

Create a service interface XML document that describes your service. Use the codegen.jar tool provided in the tools subdirectory of the AREG SDK to generate the necessary code and link it as a static library.
Extend the Stub object(s) to implement the Service Provider(s) and extend the Client Base object(s) to implement the Service Consumer(s).
Define a model for your application by describing the threads and dependencies between service components. Load the model in the main() function. Compile and link the projects with the static library of the generated code.

For multi-threading communication, you can define a model as shown in the example:

// Example of defining a model for multithreading communication

BEGIN_MODEL("ServiceModel")
    BEGIN_REGISTER_THREAD("ProviderThread", NECommon::WATCHDOG_IGNORE)
        BEGIN_REGISTER_COMPONENT("ServiceProvider", ServiceComponent)
            REGISTER_IMPLEMENT_SERVICE(NEHelloService::ServiceName, NEHelloService::InterfaceVersion)
        END_REGISTER_COMPONENT("ServiceProvider")
    END_REGISTER_THREAD("ProviderThread")
        
    BEGIN_REGISTER_THREAD("ConsumerThread", NECommon::WATCHDOG_IGNORE)
        BEGIN_REGISTER_COMPONENT("ServiceConsumer", ClientComponent)
            REGISTER_DEPENDENCY("ServiceProvider")
        END_REGISTER_COMPONENT("ServiceConsumer")
    END_REGISTER_THREAD("ConsumerThread")
END_MODEL("ServiceModel")

int main(void)
{
    Application::initApplication();
    Application::loadModel("ServiceModel");


    Application::waitAppQuit(NECommon::WAIT_INFINITE);
    Application::unloadModel("ServiceModel");
    Application::releaseApplication();
    return 0;
}

For multi-processing communication, please refer to the Hello Service! document for detailed instructions on splitting the model into two parts.

These steps will guide you in creating your own services using the AREG SDK.

Configure Multicast Router

Click to show / hide Configuring the Multicast Router.

To configure the Multicast Router, you need to adjust the areg.init file. Open the file and modify the following properties to specify the IP address and port of the mcrouter:

router::*::service          = mcrouter      # The name of the router service (process name)
router::*::connect          = tcpip         # The list of supported communication protocols
router::*::enable::tcpip    = true          # Communication protocol enable / disable flag
router::*::address::tcpip   = 127.0.0.1     # Protocol specific connection IP-address
router::*::port::tcpip      = 8181          # Protocol specific connection port number

Please note that the Multicast Router is only necessary for applications that provide or consume Public services (multiprocessing applications). If your application uses only Local services (multithreading applications), you can ignore the Multicast Router configuration.

The Multicast Router can run on any machine with a General Purpose Operating System (GPOS). It establishes a network between connected applications and is responsible for routing messages between programmable service components.

Configure Logging

Click to show / hide Configuring Logging.

To configure logging for applications based on the AREG framework, you can compile them with or without logging. If you choose to compile with logging, you'll need to configure the areg.init file to set logging scopes, priorities, and file names.

Here's an example configuration for the log file:

log::*::target              = file                          # Log targets: remote, file, debug output, database
log::*::enable::file        = true                          # File logging enable / disable flag
log::*::file::location      = ./logs/%appname%_%time%.log   # Log file location and masks

log::mcrouter::scope::*                 = NOTSET ;                         # disables logs for mcrouter.
log::my_app::scope::*                   = DEBUG | SCOPE ; # enables all logs of my_app
log::my_app::scope::ignore_this_scope   = NOTSET ;        # disables logs for a certain scope in my_app
log::my_app::scope::ignore_this_group_* = NOTSET ;        # disables logs for a certain scope group in my_app

Please refer to the AREG Logging System Wiki page for detailed instructions on compiling applications with enabled or disabled logging, starting and stopping logs during runtime, creating and using logging scopes, and configuring the areg.init file.

💡 By default, the areg.init files are located in the config subfolder of the binaries.
💡 To enable logging for all applications, use log::*::scope::* = DEBUG | SCOPE ;.
💡 Currently, logging is only possible in the file.

[ ↑ Back to top ↑ ]

Pipeline

The goal of the AREG SDK is to provide a lightweight, self-sufficient development and testing system to help developers create complex applications for Desktop, Embedded, and IoT edge devices in a shorter amount of time (30-50% faster).

Here are the planned features of the framework:

Multi-channel and multi-protocol communication.
Logging service to collect logs in the network.
Internet (web) category services.

And here are the planned tools:

Service interface designer.
Interactive log viewer.
Service testing and data simulation tool.

The development of the AREG SDK tools is currently underway in a separate repository called AREG SDK Tools, which is a part of the larger AREG SDK project.

[ ↑ Back to top ↑ ]

Use cases and benefits

The AREG SDK is a dynamic and flexible tool that offers a vast array of use cases for multithreading and multiprocessing application development. The communication engine has been successfully used in a variety of scenarios, from practical examples to real-world applications. Its proven capabilities and potential for success make it a popular choice for developers seeking to create intelligent devices for embedded and IoT applications.

To explore the various use cases of the AREG communication engine, including distributed solutions for embedded applications, driverless devices, real-time solutions, digital twins, and simulation and tests, refer to the AREG Engine Use Cases and Benefits document.

[ ↑ Back to top ↑ ]

Examples

There are various examples to demonstrate features of the AREG communication engine. The examples are part of SDK and are listed in the examples/README.md document.

[ ↑ Back to top ↑ ]

License

AREG SDK is under free open source Apache License Version 2.0. However, AREG SDK can be commercially licensed, which includes the commercial support, full rights to create and distribute software without open source license obligations. For commercial license, support or additional information, please visit Aregtech website or contact info[at]aregtech[dot]com.

[ ↑ Back to top ↑ ]

Call to action

Join our project and provide assistance by:

Checking out the list of open issues where we need help.
If you need new features, please open a new issue or start a discussion.
When creating a pull request, kindly consider the time it takes for reviewing and testing, and maintain proper coding style.
If you require invoiced commercial support or training, or if you wish to support AREG SDK commercially, kindly contact us at info[at]aregtech[dot]com.

Please repository to encourage contributors and helps to grow our community. Contact us for the feedback or new ideas.

[ ↑ Back to top ↑ ]

Thank you all!

Special thanks to all contributors and supporters that starred this repository.

Our amazing contributors:

Our supportive stargazers:

Do you like this project? Please join us or give a ⭐. This will help to attract more contributors.
Do you have an idea or found a bug? Please open an issue or start a discussion.

[ ↑ Back to top ↑ ]

Follow us at