Support multiple files for configuration

[Timothée GERMAIN]

Hi,

It would be nice be able to read configuration or at least parts of the configuration (scrapers, sinks and label mainly) from multiple files or a directory.

Why ?

If I want to deploy beamium to scrape multiple apps I need to centralize all configuration from all the things I want to scrape in the same configuration file.

It is not ideal since I need to rely on things like ansible assemble or puppet fragments to produce beamium configuration file. We have puppet ordering issues using puppet fragments for the records.

It would be cleaner for me to deploy :

• beamium as a separate role which install beamium and maybe set some default parameters (like the parameters section of beamium conf)
• Modify role related to apps I want to scrap to add configuration for beamium. The scrapers part of beamium conf.

I think it could also ease the work on #51 but it is a sweet side effect.

Proposal

/etc/beamium.yaml

[...]
sinks:
  sinkdir: /etc/beamium.d/sinks/
[...]

/etc/beamium.d/sinks/my-sink.yaml

[...]
  scraper1:                            # Source name                  (Required)
    url: http://127.0.0.1:9100/metrics # Prometheus endpoint          (Required)
    period: 10000                      # Polling interval(ms)         (Required)
    format: prometheus                 # Polling format               (Optional, default: prometheus, value: [prometheus, sensision])
    metrics:                           # Filter fetched metrics       (Optional)
      - node.*                         # Regex used to select metrics (Required)
    headers:                           # Add custom header on request (Optional)
      X-Toto: tata                     # list of headers to add       (Optional)
      Authorization: Basic XXXXXXXX
[...]

[Steven Le Roux]

Seems like a valid use case.

Composition in the configuration should be made more generic that just for sinks. Besides, the given example shows a custom scraper.

parameters: # Parameters definitions (Optional)
  conf-dir: /etc/beamium/conf.d       # Beamer config files directory                    (Optional, default: null)

conf-dir would be skimed recursively, so that you could organise it like this :

/etc/beamium# tree .
.
└── config.yaml
├── conf.d
    ├── app1.yml
    ├── app2
    │   └── app2.yml
    ├── app3
          ├── sink.yml
          └── scraper.yml

I agree this rework could take #51 into account.