2d-optimization flipped axis
Webbah opened this issue · comments
we have used this toolbox as an example in our application (https://github.com/upb-lea/openmodelica-microgrid-gym), where we will design a current controller (standard PI; Kp/Ki) for an inverter.
expected behavoiur
When fliping the dimensions of our two dimensional optimization space, we would expect the ploted results should only flip the axis.
Observed behavior
However, when observing the point marked in green, the values change!
what we investigated
bounds are flipped accourdingly
Ideas
We have the impression, that the first dimension is explored much better than the second dimension. in the left plot the y values never exeed 0.06 even though they are deemed save in the right experiment. Similarly in the right plot, it never dares to try above 150, but in the left it is explored and also save. Therefore we assume it is related to the handling of expanders.
Do you have any hints on how to investigate it further. We are not certain that it is a bug, but the anti-symmetry seems odd.
Which part of the code is this based on? If this is using a discretization of the space my guess would be that it's related to how the optimizer (numpy.argmax) breaks ties: by returning the first parameter that yields the maximum value. That could lead to additional exploration in one dimension especially in the beginning. Though without further details on the code it's difficult to diagnose the problem - there are too many variables in practice, ranging from model-missspecification and hyperparameter optimization to the way expanders are constructed.
Though overall the plots look reasonably similar to me. Does the lack of exploration persist when selecting more data points?
In order to rule out that it is caused by exploration, we investigated more episodes.
As can be seen in the figures, the result looks similar but still differ in detail.
Seems like lower dimensions (x-axis) are preferred over higher dimensions (y-axis).
However, that is in alignment to your previous argument.
So besides some final doubts we are satisfied and thanks for your fast answer.
(Flipped screenshot, same simulation setup, changed parameter order given to the optimizer)