Comparison between past benchmarks
zuliani99 opened this issue · comments
Make attention to line 165 of utils.py:
if (
collections.Counter(cls) == collections.Counter(dfs_class)
and collections.Counter(reg) == collections.Counter(dfs_reg)
and collections.Counter(pip) != collections.Counter(options_end)
)
This allow comparison only if there are the same classification and regression dataframe and if the end lifetime is different, note that now the end lifetime is the actual time consumed for computation. Which is also rounded to 3 significant digits.
So decide in a short time what logic to implement in this part of the code
Changed in:
if (
collections.Counter(cls) == collections.Counter(dfs_class)
and collections.Counter(reg) == collections.Counter(dfs_reg)
and collections.Counter(time_limit) != collections.Counter(options_start)
)
Now I can compare benchmarks with the same dataframes but with different starting limit time
At the moment it seems to be ok. If the problem isn't resolved i'll reopen this issue