Support alternate CloudWatch configurations for monitoring Data Prepper
dlvenable opened this issue · comments
Is your feature request related to a problem? Please describe.
Data Prepper supports CloudWatch as a metric destination for its own metrics. The data-prepper-core project includes a cloudwatch.properties file.
It is good that Data Prepper is pre-configured for CloudWatch so that no additional configuration is needed.
However, the current approach provides some difficulties:
- It is difficult to override these configurations
- When creating a custom Data Prepper distribution, the file in this jar may conflict with other similar files that teams provide.
Describe the solution you'd like
Move the cloudwatch.properties file out of data-prepper-core.jar. Instead, include it in the config directory of Data Prepper at config/cloudwatch.properties. With this, users can change this file and get their own custom configurations.
Describe alternatives you've considered (Optional)
There may still be some value in including the cloudwatch.properties file in the jars. If so, we may wish to move it into data-prepper-main.jar since that jar is more opinionated on how Data Prepper is run.
Additional context
See #4464 for another file that is in data-prepper-core.jar and causing difficulty for custom distributions.