tag-like filtering in custom fields

[Iskander14yo]

Proposal Summary

Hello!
I'd like to ask the developers to add option to make values of parameters be tag-like. As you know, set_parameters casts parameter arguments to a string, which is not very convenient when you want more flexible filtering, as in tags. Yep, one can extract these params and convert them back to list, but this cannot be done in web-interface.

Motivation

Flexibility for analytics and their managers who don't want to mess with code but rather interact with pretty web-interface.

Related Discussion

Didn't find related discussion but maybe I am bad at searching.

[Iskander14yo]

Follow-up: I understand that this issue leads to much work, therefore - is there any workaround for now?

[Noam Wasersprung]

Thanks for proposing @Iskander14yo. Just to be sure what you mean - Can you give a concrete example of the use-case you are considering?

[Iskander14yo]

As an example, one may consider a list of datasets used by the model, a list of projects (yes, ClearML provides the possibility of splitting tasks into projects, but there is a lot of cross-project research in our teams).
In general, any list-like attributes of task that would be much more convenient to filter the way you did for tags: model types, development stage, arch distinctions and features, priority and many others. It's not very convenient to store it all in one tags field.

[Noam Wasersprung]

Thanks for the outline @Iskander14yo. An example of a set_parameters() call and a couple of filters for it would really help clarify the example.

[Iskander14yo]

When I set parameters that I want to see in table of experiments I do following:

config = {
    'batch_size': 64,
    'epochs': 5,
    'lr': 0.01,
    'datasets': ['001', '003', '004'], # this
    'projects': ['LLM', 'GPT', 'AGT', 'AnotherProject'], # and this
}
task.set_parameters(**config)

When get task parameters, I see them set this way:

{'General/batch_size': '64', 'General/epochs': '5', 'General/lr': '0.01', 'General/datasets': '["001", "003", "004"]', 'General/projects': '["LLM", "GPT", "AGT", "AnotherProject"]'}

As you see, datasets and projects are strings, not lists. Therefore when trying to filter on these parameters, I can filter on whole string only, not on elements of list compared to attributes like tags.

As I see it, i would be nice to be able to pass data types such as lists as task parameters in order to be able to conveniently filter, as in tags, user or status. I can pass such parameters through tags (which suggests that my proposal is possible), but then there will be a mess in the tags, which I would like to avoid so that the table can look like a full-fledged good dashboard.

[Iskander14yo]

I could try to make changes to the clearml code myself and send a PR, but I don't have time for that yet.
You can close the issue and I will reopen it when (if) I contribute. Does it make sense?

[Noam Wasersprung]

Thanks for all the details @Iskander14yo. We can definitely keep the issue open so other users can voice out.
Indeed for the time being the only way to get UI filtering like you are looking for would be to drop all these items at tags.
Hyperparameters are intrinsically treated as strings, and this will not change any time soon.
We do have a long term feature we're looking at for custom tag fields (i.e. you can define additional tag fields so as not to clutter all your labels in one field). An interim solution might become available through providing partial string match for the hyperparameter filter - Do you think that would be useful?

BTW - are these actually parameters, or basically metadata you are recording?

[Iskander14yo]

Custom tag fields are indeed what I am asking for. So users have tags-like functionality on user-defined fields. Suggested interim solution also seems to be a good option.

Yes, these are not parameters but rather meta, as you said, that I'd like to store and refer in the future.