General

grid2vec is a rewrite of grid2op that aims to address some of the weaknesses we encountered while using grid2op.

Most notably, grid2vec is designed to

Work with multiple busbars per substation, with a custom substation affinity file
Compute n-1 loadflows natively
Be built around the (future) prospect of a batched solver which can compute a large batch of powerflows at the same time. Because of this, offer a vectorized interface
Be built around the (future) prospect of running the entire environment loop on GPU.
Offer switch and phase-shift transformer actions.
Offer extensive masking capabilities, in case the pandapower grid holds a lot of elements that are irrelevant

Other features of grid2op have been deemed irrelevant for our use-case and were not implemented:

Redispatch actions
Storage actions
Opponent modelling (done through full n-1 analysis)
Disconnect on overload (in the real world, unmitigated overloads will be subject to redispatch, minimize the amount of overloads)
Alarms (It's clear from the loadflow results whether an action is a good one)
Forecasts (We will only get the forecast data, no ground truth, so train directly on forecasts)
...

Engineering decisions

Our engineering builds on the following assumptions:

There will be capable batched load-flow solvers in the near future that leverage the potential of running many, similar computations to gain ca 2 OOM speedup
These solvers will benefit from as large batch sizes as possible, the more the merrier
These solvers might utilize GPUs

GPU compatibility of the environment

The environment is written in jax and is jit compatible, so it can be easily run on a GPU. Currently that doesn't make sense because no GPU solver is integrated, so you won't actually feel a speedup from using grid2vec yet. As soon as a batched and GPU-native solver is integrated, this version should hopefully outperform grid2op.

Grid env split

We decided to implement a split of the environment into a read-only portion that is only loaded once and stays constant throughout the entire optimization (grid) and a part that is changing in every optimization step (env). The main reason for this is that jax benefits from static variables and that the chronics can be held in ram in a central place. The static grid dataclass can be held in shared memory or be loaded before a fork in multi-process environments. In a vectorized environment, all instances can access the same central chronics data.

It turns out that we don't have a native GPU solver yet, so we can't tap into the potential performance benefit of the grid-env split. Currently it's actually worse because we have to load all chronics at startup and you need enough RAM on your machine to hold them all.

Grid file format

The grids are stored in a file format which is as close to grid2op as could be. It's a folder with mainly a grid.json pandapower net in it, a chronics folder and extra data in the form of numpy arrays:

grid.json: Holds the base pandapower network
Action masks:
- line_controllable.npy: A mask of shape (len(net.line),), which lines should be part of the line_status action space (line connection/disconnection). If this file is missing, all lines are considered action space.
- trafo_controllable.npy A mask of shape (len(net.trafo),), which trafos should be part of the trafo_status action space (trafo in service/out of service). If this file is missing, all trafos are considered action space
- trafo_tap_controllable.npy A mask of shape (len(net.trafo),) which trafos can have their taps set as part of the action space. If missing, all trafos with valid tap information in the pandapower file are considered action space.
- trafo3w_tap_controllable.npy A mask of shape (len(net.trafo3w),) which trafo3ws can have their taps set as part of the action space. If missing, all trafos with valid tap information in the pandapower file are considered action space.
- substation_affinity.npy The substation affinity file, for more information consult the Substation Affinity section of this document.
N-1 Masks:
- line_for_nminus1.npy: A mask of shape (len(net.line),) which lines should be part of the nminus1. If this file is missing, no lines will be part of the n-1 computation.
- trafo_for_nminus1.npy: A mask of shape (len(net.trafo),) which trafos should be part of the nminus1. If this file is missing, no trafos will be part of the n-1 computation.
- trafo3w_for_nminus1.npy: A mask of shape (len(net.trafo3w),) which trafo3ws should be part of the nminus1. If this file is missing, no trafo3ws will be part of the n-1 computation.
- bus_for_nminus1.npy: A mask of shape (len(net.bus),) which buses should be part of the nminus1. If this file is missing, no buses will be part of the n-1 computation.
Reward masks
- line_for_reward.npy: A mask of shape (len(net.line),) which lines should be part of the reward formulation. If this file is missing, all lines are part of the reward.
- trafo_for_reward.npy: A mask of shape (len(net.trafo),) which trafos should be part of the reward formulation. If this file is missing, all trafos are part of the reward.
- trafo3w_for_reward.npy: A mask of shape (len(net.trafo3w),) which trafo3ws should be part of the reward formulation. If this file is missing, all trafo3ws are part of the reward.
A chronics folder. Chronics are saved in the same format as grid2op, meaning there are four files for each chronic.
- load_p.csv.bz2 is a matrix of shape (n_timestep, n_load) and corresponds to the net.load.p_mw values
- load_q.csv.bz2 is a matrix of shape (n_timestep, n_load) and corresponts to the net.load.q_mvar values
- prod_p.csv.bz2 is a matrix of shape (n_timestep, n_gen) and corresponds to the net.gen.p_mv values
- prod_v.csv.bz2 is a matrix of shape (n_timestep, n_gen) and holds vn_kv values for the generators. Pandapower requires the voltages in vm_pu though, so they need to be converted back with net.gen.vm_pu = prod_v[timestep] / net.bus.loc[net.gen.bus]["vn_kv"].values

Substation Affinity

The substation affinity file is a file denoting which element can be assigned on which busbars. It is a matrix of shape (len_topo_vect, max_busbars_per_substation) with bus-IDs as content. The topo-vect concept is a concept borrowed from grid2op - in grid2op it's a vector of size n_line * 2 + n_load + n_gen + n_storage where every index holds the bus the element is assigned to. In grid2vec, we borrow this concept with the following order:

line, from side
line, to side
trafo, hv side
trafo, lv side
trafo3w, hv side
trafo3w, mv side
trafo3w, lv side
gen
load
sgen

You can use grid.topo_vect_to_pandapower to get a lookup table for the topo vect.

The substation affinity file has the same width, but has a second dimension max_busbars_per_substation. If the most busses per substation anywhere in the grid is 4, this is 4. Now, not every element has 4 different busses it can be assigned to. In that case, the special bus -1 should be chosen for masking away those entries.

For a network with 1 line and a generator and a load, an example substation affinity file could look like this:

[[ 1,  2, -1, -1],  # The line from-side can be on either bus 1 or 2
 [ 1,  2,  3,  4],  # The line to-side can be on bus 1, 2, 3 or 4
 [ 2,  3, -1, -1],  # the generator can be on bus 2 or 3
 [ 4, -1, -1, -1]]  # the load can only be on bus 4 and cannot be moved

If there are invalid combinations that you want to mask out (e.g. both line ends on the same busbars), use action dumps to list all valid actions.

Masking

There are multiple masks,

for the action space _controllable.npy
for the n-1 computation _for_nminus1.npy
for the reward computation _for_reward.npy
Not implemented yet, for the observation space _for_observation.npy

These allow to have granular control over which parts of the net should be part of the action space. As we run our PoC on data of the entire european net but only control the german section of it, this is necessary.

The n-1 masks are stored in an nminus1_definition as defined in nminus1_definition.py. The action space masks are mainly processed in env.py but loaded along the grid. The reward masks are processed in env.py but loaded along the grid.

michaeldorner / test