Bigomby / rbforwarder

Extensible, modular and easy to use tool for process data on a pipeline

rbforwarder

rbforwarder is an extensible tool for processing data asynchronously. It allows you to create a custom pipeline in a modular fashion.

You can use rbforwarder to build a pipeline that decodes a JSON, filter or add fields, encode the data again to JSON and send it using multiple protocols HTTP, MQTT, AMQP, etc. It's easy to write a pipeline component.

Built-in features

• Support multiple workers for each components.
• Asynchronous report system. Get responses on a separate goroutine.
• Built-in message retrying. rbforwarder can retry messages on fail.
• Instrumentation, to have an overview of how the application is performing.

Components

• Send data to an endpoint
  • MQTT
  • HTTP
  • Kafka
• Decoders / Encoders
  • JSON
• Utility
  • Limiter
  • Batcher (supports deflate)

Road Map

The application is on hard development, breaking changes expected until 1.0.

Milestone	Feature	Status
0.1	Pipeline	Done
0.2	Reporter	Done
0.3	HTTP component	Done
0.4	Batcher component	Done
0.5	Limiter component	Done
0.6	Instrumentation	Pending
0.8	JSON component	Pending
0.9	MQTT component	Pending
1.0	Kafka component	Pending