`False` as a default to 'switch off' a constraint can lead to unexpected results
brynpickering opened this issue · comments
Problem description
As discovered in #292, if you have mixed data in an xarray DataArray, where one dtype is boolean, it is not possible to save it to file. These mixed dtypes come from parameters that can be either a finite numeric value or False. If False, associated constraints are effectively switched off. However, if the value is 0, this is considered a numeric value and the constraint is applied with that value (e.g. energy_cap_equals goes from being switched off to effectively stopping the technology from having any capacity).
To save a mixed bool-numeric dtype to file, the most straightforward approach is to convert it to a pure numeric dtype. However, when loading that data into a calliope model and running it, what were once False values are loaded back in as 0. The result of running this model will be different to running the original.
Getting here is something of an edge case:
model = Calliope.Model(...)
model.run()
# There are unlikely to be any mixed dtypes here, so no problem will be caused
model.to_netcdf(...)
# This will give the same result as the initial run,
# but inputs are returned from the Pyomo object,
# with default values filled in (`NaN`s now become e.g. `False`).
new_model = model.backend.rerun()
# To succeed here, the new mixed dtype DataArrays need to be converted to numeric
new_model.to_netcdf(...)
# This result will now be different to the initial run,
# since the underlying input data has had its mixed dtypes
# converted to pure numeric for saving to NetCDF.
new_model.run(force_rerun=True) This bug is silent but deadly. It could be introduced without following the abovementioned steps (e.g. just defining False for a parameter explicitly in the input YAML). It's clear that this shouldn't happen. We've noted the poor defaults in the past (#260) but need to really address it in some way in v0.6.8.
I would aim to have all equals constraints default to None or NaN and actively look for that when deciding whether to set a constraint in the Pyomo backend. This requires changing the Pyomo parameter domains (#313) to Any as None/NaN don't fit into e.g. NonNegativeReals. There is no major issue here, except that a user can then update the Pyomo backend parameters with any value they wish, rather than the set of values that are actually 'allowed' (e.g. could put a negative energy_cap_max in). The model would then be infeasible, which is better than silently solving the model incorrectly.
Calliope version
v0.6.8-dev
Fixed in 7ef0ac6