Processing of Core TSO Grid Data
As part of the extension of Flow-Based Market Coupling (FBMC) from FR-DE-BE-NL-LU eastwards this year to encompass the "Core" European countries (FR-DE-BE-NL-LU-PL-CZ-SK-SI-HU-RO-HR-AT), the grid operators published in December 2021 a Static Grid Model covering these Core countries:
https://www.jao.eu/static-grid-model
"The aim of the publication of the SGM is to allow market participants to do market analyses in order to enhance efficiency in the market."
Unlike previously-published models, this has a uniform format, thermal ratings for different seasons, non-obscured substation names and all electrical parameters and lengths. (E.g. unlike the previous data tables on TSO websites which were different for each TSO, or the TYNDP and BNetzA grid models which often had cryptic or obscured substation names which hinders georeferencing).
Installation
conda env create -f environment.yaml
Running
snakemake -j 1 process_data
Helpers for Manual Corrections
Openinframap.org
import webbrowser
import pandas as pd
fn = "outputs/locator-results.csv"
df = pd.read_csv(fn, index_col=0)
i = 0
i += 1
name, x, y, _ = df.iloc[i]
print(name)
webbrowser.open(f'https://openinframap.org/#15/{y}/{x}')
Match substation names with OpenStreetMap
See ./OSM-locator.
- Download PBF files with OSM data (all NUTS 1 files for one country for better performance)
- Get the information about the EHV substations in OSM
- Compare the names the the OSM data and Core TSO with fuzzywuzzy (it gives a list of the 5 best matches)
- Check manually if the choices are correct:
- If is not the first match, see if it is one of the others (almost always is the second)
- If there is data missing in OSM: Add it in OSM and return to step 1
- A couple of dictionries make the linguistic adjustments that maximize the matching names.
- Eventually the only substations not found are either located in another country or still under construction.
- All the substaions in the Core-TSO data are match to a substaion in OSM (except for the the ones located in a different country or under construction).
- Updating the OSM is the most time consuming step, but is easy to do and worth it for the community.
- In OSM sometimes there is more than one substation with the same name: Just take the first one (for now).
- A 100 % match of the names is impossible (more fancy fuzzywuzzy functions were tested), but the manual adjustments is minimal.
- The missing data from other countries and Tie Lines and Trafos, could be allocated later
Convert locator results pd.DataFrame to YAML
import pandas as pd
import yaml
fn = "outputs/locator-results.csv"
df = pd.read_csv(fn, index_col=0)
d = df.set_index('name')[["x", "y"]].T.to_dict()
with open('my-corrections.yaml', 'w') as yaml_file:
yaml.dump(
d,
yaml_file,
default_flow_style=False,
allow_unicode=True,
sort_keys=False
)
License
The code is distributed with an MIT license.
The source data has no license. Use at own risk.