Formatting of results column
sjoerdvansteenkiste opened this issue · comments
Currently floats that are displayed in the results column are not clipped up to a fixed decimal point. I.e. currently I see the following:
0.773208737373352,0.8946895003318787,2824.651123046875
and I would suggest something along the lines of
0.77,0.89,2824.65
Moreover, when saving multiple results in a list (as is the case in the above example) it would be nice to add some spacing.
0.77, 0.89, 2824.65
Hi,
thank for your idea. I originally expected the result field to contain only a single number but now I see it's not enough.
- Regarding clipping, the question is how to determine the number of digits to show.
- Spacing is obvious (for arrays), some formatting would be nice.
What about adding the result as a browsable element similarly to info or config in the run detail view?
I would have similar requirements like @sjoerdvansteenkiste , because I usually compute one or more metrics in addition to the cost-function. Unfortunately, It looks like sacredboard can only display a single scalar value in the list right now. Or am I missing something?
I tried to return a dictionary instead of a scalar, but this is only displayed as "object Object" and it is not possible to see values contained in the dictionary. Even when I return a dictionary with just a 'optimization_target' value, which is recommended when using labwatch also leads to the problem that no result value is shown in the table.
Returning multiple values as a list is a bad workaround in my opinion, because sorting of the "Results" column might then not be possible anymore.
My suggestion:
- when only a single value is returned
- store the value in the results column
- when a dictionary is returned:
- store the key "optimization_target" in the result column (to be compatible with sacred's labwatch extension)
- create a new result column for every other key in the dictionary
- optional: when a list is returned:
- Create a column for every value: Result, Result2, Result3, ...
This would have the advantage than you can sort the results according to every result. As an example, an experiment could return the following:
- loss
- metrics (accuracy, recall, precission)
- steps / epochs (how long did the training run until it was stopped due to the stopping criteria)
I am also interested in such a feature.
@bsautermeister: This will make the number of columns dynamic and it could be huge if you return too many values. Even though sorting regarding results would be nice.
The problem I see as well, is that not all your experiments might have the same result dictionary, so many columns would be empty.
Generally, I think that issue #24 does answer your problem and would be more generic.
The results should then be saved as @chovanecm proposed above:
What about adding the result as a browsable element similarly to info or config in the run detail view?
Ok, so this looks like something that is really wanted.
For now, I would stay with the detail view similar to config view.
May I ask some of you to attach an experiment with such result structure to this ticket?
Should be done together with #24
At least the first step is done: result is now listed in the detail view and can be rendered properly for both complex and scalar values.