[Jsonnet template] Support for multiple prometheuses statefulsets.

[Riccardo Scartozzi (risca)]

What is missing?

I'm looking forward for a way to deploy multiple prometheuses statefulsets via jsonnet and the prometheus-operator.

Why do we need it?

If it could be possible to have multiple prometheuses instances groups, then it would be easy to arbitrarily shard the monitoring stack. This is a little different from the current automatic sharding option since this way I could have some instances dedicated only to certain targets/rules via proper namespace/label selector.

I would expect some benefits such as:

• easier scaling of monitoring workload;
• management improvement such as instances dedicated to different teams or different operational profiles.

Environment

• kube-prometheus version:

  Latest: e7fa2f0b3c7606296dba118cc1759e8055f7215b

Anything else we need to know?:

Obviously the same consideration about the sharding and replica are still valid about proper metrics inspection and rules evaluation (such as prometheus federation or thanos backend).
It could also still be desired to have the option of the automatic sharding inside of the specific prometheus statefulset itself.