rocknsm / rock

Automated deployment scripts for the RockNSM network hunting distribution.

Home Page:http://rocknsm.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Update Kafka to add more partitions

dcode opened this issue · comments

Kafka scales by adding partitions. By default, we currently only create a single partition per topic. The number of partitions should be a multiple of the number of consumers. In the rocknsm stack alone, that's equivalent to the number of logstash consumer threads per topic which defaults to 1. We should increase the number of partitions to maybe a dozen by default. It doesn't hurt anything and only provides the capability to scale as needed.