tinkoff-ai / etna

ETNA – Time-Series Library

Home Page:https://etna.tinkoff.ru

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Vectorize metric computation

Mr-Geekman opened this issue Β· comments

πŸš€ Feature Request

Currently our metrics are calculated in per-segment fashion. We could fix it to optimize metric computation speed.

Proposal

Add new parameter to Metric: metric_fn_signature. It could have two possible values:

  • array_to_float: default, it expects two vectors and returns float,
  • matrix_to_array: it expects two matrices and returns vector.

Rework functional metrics to work with new matrix mode by checking the input shape:

  • mape
  • smape
  • sign
  • max_deviation
  • wape

Rework all current metrics (except for interval metrics) to work in matrix mode. For metrics based on sklearn we should use multioutput parameter.

To discuss:

  • Could we add more metrics? For example, we could follow this logic for Sign, Width.
  • Setting multioutput parameter manually could interfere with self.kwargs. What are we going to do about it?

Test cases

  • Make sure current tests pass
  • Fix test_metrics_values
  • Rework tests for functional metrics

Additional context

Related: #1338.