Add global and by tag logging functions

[Tom]

Hi !
I am in a situation where I want to update and reset several tags (or all metrics) at the same moment, and I haven't found a way to do it without calling log_and_reset for each of these metrics.

• Have I missed something in the logger?
• If it isn't already available, would you consider adding this feature? (If so, I can send a PR)

[Leonard Berrada]

Hi, you're correct that this is not implemented in the current code. In order to be able to log several or all metrics, one possibility would be to introduce some regex in the log_with_tag function: this would allow to generalize the behavior without breaking the current API nor adding extra methods. Would that be useful in your case?

[Tom]

It would solve the problem of updating several tags yes, I guess giving a list of tags would be another simple solution. However, it wouldn't help for the reset part.
I've implemented another approach, adding a reset flag to the log_with_tag function and adding a log_all_metrics function, allowing a third level of granularity in the metric selection:

1. Individual metric, using log_metric and log_and_reset_metric
2. Select metrics by tag, using log_with_tag (which could be used with a regex or list for several tags)
3. Select all metrics, using log_all_metrics

The two last methods having a reset flag, which could later be added in the first method to standardize the interface.
While this solution doesn't break the current API, it adds an extra method. What's your opinion?

[Leonard Berrada]

Using a reset parameter is a very good idea indeed, it would avoid the need for the log_and_reset method.
As for the log_with_tag method, I'd rather preserve the current type of arguments and not switch it to iterables.
The log_all_metrics method is not necessary if a simple regex can retrieve all tags.

[Tom]

I just noticed that using a regex will allow to keep the same argument types, but will change the behavior in some cases:
For example, if an experiment has the tags train and train_b. In the current version, a call to log_with_tag(tag='train') will only log train tag while interpreting the argument as a regex will result in logging both train and train_b (as regex tries to match from the beginning).
In this case, the user should call log_with_tag(tag='train$'),
I'm not sure if there's a clean solution allowing multi-tag logging without breaking the current API nor adding extra methods.

[Leonard Berrada]

Ok, regex might complicate things, what I actually had in mind was matching with unix-style wildcards. It looks easy enough to use and would keep the current behavior: in your example log_with_tag("*") would log everything, log_with_tag("train*") would log train and train_b, and the current behavior would be preserved. Does that sound reasonable?

[Tom]

Ok, this seems indeed to be a good solution.

[Leonard Berrada]

Solved by #9