Bigomby / events-counter-cpp

Simple utility that for counting messages in a Kafka topic

events-counter

• Overview
• Install
• Usage
  • License files
  • Configuration
• Roadmap

Overview

events-counter is a simple utility that can be used for accounting messages in a Kafka topic. It supports accounting by number of messages or by total number of bytes (regardless the numer of messages). Messages are expected to be on JSON format when counting messages.

Install

./configure
make
make install

This will generate a ready to use binary.

Usage

./events-counter -c <config.json>

In the config file you can specify some filters. For example, you can count messages with a specific uuid.

License files

It's possible to compile events-counter with support for license files i.e., signed configuration files. When the application is compiled with licensed mode enabled it will try to verify the configuration file and wont start if the verification process fail.

Configuration

{
    "uuids": ["2", "4", "7"],
    "monitor_config": {
        "timer_seconds": {
            "period": 86400,
            "offset": 40800
        },
        "read_topic": "rb_counters",
        "write_topic": "rb_limits",
        "rdkafka": {
            "read": {
                "topic.auto.offset.reset": "smallest",
                "metadata.broker.list": "kafka:9092",
                "enable.auto.commit": "false",
                "group.id": "monitors"
            },
            "write": {
                "metadata.broker.list": "kafka:9092"
            }
        },
        "limits": [{
            "uuid": "2",
            "limit": 370000
        }, {
            "uuid": "4",
            "limit": 370000
        }, {
            "uuid": "7",
            "limit": 80
        }]
    },
    "counters_config": {
        "update_interval": 1,
        "json_read_uuid_key": "sensor_uuid",
        "read_topics": ["rb_flow"],
        "write_topic": "rb_counters",
        "rdkafka": {
            "read": {
                "metadata.broker.list": "kafka:9092",
                "group.id": "counters"
            },
            "write": {
                "metadata.broker.list": "kafka:9092"
            }
        }
    }
}

Roadmap

Version	Feature	Status
0.2	Counter module	✅
0.3	Config module	✅
0.4	Monitor module	✅
0.5	License module	🚧