Use the _index_template API for metrics templates

[Jason Bryan]

Today, Rally uses the legacy put_template API for creating index templates on the metrics store. Rally should instead use put_index_template (composable templates) when creating metrics index templates so it is using the most current API and to advance it one step toward sending metrics to a serverless project.

2023-12-01 23:20:04,259 -not-actor-/PID:1624 esrally.rally ERROR Cannot run subcommand [race].
Traceback (most recent call last):
  File "/home/esbench/rally/esrally/rally.py", line 1183, in dispatch_sub_command
    race(cfg, args.kill_running_processes)
  File "/home/esbench/rally/esrally/rally.py", line 931, in race
    with_actor_system(racecontrol.run, cfg)
  File "/home/esbench/rally/esrally/rally.py", line 961, in with_actor_system
    runnable(cfg)
  File "/home/esbench/rally/esrally/racecontrol.py", line 399, in run
    raise e
  File "/home/esbench/rally/esrally/racecontrol.py", line 396, in run
    pipeline(cfg)
  File "/home/esbench/rally/esrally/racecontrol.py", line 72, in __call__
    self.target(cfg)
  File "/home/esbench/rally/esrally/racecontrol.py", line 335, in benchmark_only
    return race(cfg, external=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/rally/esrally/racecontrol.py", line 293, in race
    raise exceptions.RallyError(result.message, result.cause)
esrally.exceptions.RallyError: Traceback (most recent call last):
  File "/home/esbench/rally/esrally/metrics.py", line 110, in guarded
    return target(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/.local/lib/python3.11/site-packages/elasticsearch/_sync/client/utils.py", line 414, in wrapped
    return api(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/.local/lib/python3.11/site-packages/elasticsearch/_sync/client/indices.py", line 2891, in put_template
    return self.perform_request(  # type: ignore[return-value]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/.local/lib/python3.11/site-packages/elasticsearch/_sync/client/_base.py", line 390, in perform_request
    return self._client.perform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/rally/esrally/client/synchronous.py", line 226, in perform_request
    raise HTTP_EXCEPTIONS.get(meta.status, ApiError)(message=message, meta=meta, body=resp_body)
- elasticsearch.ApiError: ApiError(410, 'api_not_available_exception', 'uri [/_template/rally-metrics] with method [PUT] exists but is not available when running in serverless mode')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/esbench/rally/esrally/actor.py", line 92, in guard
    return f(self, msg, sender)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/rally/esrally/racecontrol.py", line 111, in receiveMsg_Setup
    self.coordinator.setup(sources=msg.sources)
  File "/home/esbench/rally/esrally/racecontrol.py", line 235, in setup
    self.metrics_store = metrics.metrics_store(
                         ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/rally/esrally/metrics.py", line 365, in metrics_store
    store.open(race_id, race_timestamp, track, challenge, selected_car, create=not read_only)
  File "/home/esbench/rally/esrally/metrics.py", line 906, in open
    self._client.put_template("rally-metrics", self._get_template())
  File "/home/esbench/rally/esrally/metrics.py", line 53, in put_template
    return self.guarded(self._client.indices.put_template, name=name, **tmpl)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/esbench/rally/esrally/metrics.py", line 208, in guarded
    raise exceptions.RallyError(msg)
esrally.exceptions.RallyError: An error [api_not_available_exception] occurred while running the operation [put_template] against your Elasticsearch metrics store on host [e185a0-715d-4e52-9359-46a9740a27c2-b3094f.es.domain] at port [443].

This works with just a few minor changes for which I intend to file a PR. We'd just need to make sure it isn't disruptive to nightlies, etc.