Giters

AgtLucas / electricitymap-contrib

A real-time visualisation of the CO2 emissions of electricity consumption

Home Page:https://www.electricitymap.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

electricityMap Slack Status CircleCI Twitter Follow

A real-time visualisation of the Greenhouse Gas (in terms of CO2 equivalent) footprint of electricity consumption built with d3.js and mapbox GL, maintained by Tomorrow. Try it out at http://www.electricitymap.org, or download the app:

Get it on Google Play Get it on the Apple Store

image

You can contribute by

You can also see a list of missing data displayed as warnings in the developer console, or question marks in the country panel:

image

Check the contributing section for more details. Join us on Slack if you wish to discuss development or need help to get started.

Tomorrow is hiring!

The company behind the electricityMap builds tech to empower organisations and individuals to understand and reduce their carbon footprint.

We're hiring great people to join our team in Copenhagen. Head over to our jobs page if you want to help out!

Frequently asked questions

How do you define real-time data? Real-time data is defined as a data source with an hourly (or better) frequency, delayed by less than 2hrs. It should provide a breakdown by generation type. Often fossil fuel generation (coal/gas/oil) is combined under a single heading like 'thermal' or 'conventional', this is not a problem.

Why do you calculate the carbon intensity of consumption? In short, citizens should not be responsible for the emissions associated with all the products they export, but only for what they consume. Consumption-based accounting (CBA) is a very important aspect of climate policy, and allows to assign responsibility to consumers instead of producers. Furthermore, this method is robust to governments relocating dirty production to neighbouring countries in order to green their image while still importing from it. We published our methodology here.

Why don't you show emissions per capita? A country that has few inhabitants but a lot of factories will appear high on CO2/capita. This means you can "trick" the numbers by moving your factory abroad and import the produced good instead of the electricity itself. That country now has a low CO2/capita number because we only count CO2 for electricity (not for imported/exported goods). The CO2/capita metric, by involving the size of the population, and by not integrating all CO2 emission sources, is thus an incomplete metric. CO2 intensity on the other hand only describes where is the best place to put that factory (and when it is best to use electricity), enabling proper decisions.

CO2 emission factors look high — what do they cover exactly? The carbon intensity of each type of power plant takes into account emissions arising from the whole life cycle of the plant (construction, fuel production, operational emissions, and decommissioning).

Is delayed data useful? While the map relies on having real-time data to work it's still useful to collect data from days/months past. This older data can be used to show past emissions and build up a better dataset. So if there's an hourly data source that lags several days behind you can still build a parser for it.

Can scheduled/assumed generation data be used? The electricityMap doesn't use scheduled generation data or make assumptions about unknown fuel mixes. This is to avoid introducing extra uncertainty into emissions calculations.

Can areas other than countries be shown? Yes providing there is a valid GeoJSON geometry (or another format that can be converted) for the area. As an example we already split several countries into states and grid regions.

How can I get access to historical data or the live API? All this and more can be found here.

Data sources

Carbon intensity calculation and data source

The carbon intensity of each country is measured from the perspective of a consumer. It represents the greenhouse gas footprint of 1 kWh consumed inside a given country. The footprint is measured in gCO2eq (grams CO2 equivalent), meaning each greenhouse gas is converted to its CO2 equivalent in terms of global warming potential over 100 year (for instance, 1 gram of methane emitted has the same global warming impact during 100 years as ~34 grams of CO2 over the same period).

The carbon intensity of each type of power plant takes into account emissions arising from the whole life cycle of the plant (construction, fuel production, operational emissions, and decommissioning). Carbon-intensity factors used in the map are detailed in co2eq_parameters.json. These numbers come mostly from the following scientific peer reviewed literature: IPCC (2014) Fifth Assessment Report is used as reference in most instances (see a summary in the wikipedia entry)

Each country has a CO2 mass flow that depends on neighbouring countries. In order to determine the carbon footprint of each country, the set of coupled CO2 mass flow balance equations of each countries must be solved simultaneously. This is done by solving the linear system of equations defining the network of greenhouse gas exchanges. Take a look at this notebook for a deeper explanation. We also published our methodology here.

Real-time electricity data sources

Real-time electricity data is obtained using parsers.  

Click to see the full list of sources

Production capacity data sources

Production capacities are centralized in the zones.json file. Values in the capacity maps are in MW.  

Click to see the full list of sources

Cross-border transmission capacity data sources

Cross-border transmission capacities between the zones are centralized in the exchanges.json file. Values in the capacity maps are in MW.  

Click to see the full list of sources

A ⇄ B: bidirectional operation, with power flow either "from A to B" or "from B to A"

A ⇉ B: unidirectional operation, only with power flow "from A to B"  

Electricity prices (day-ahead) data sources

Real-time weather data sources

We use the US National Weather Service's Global Forecast System (GFS)'s GFS 0.25 Degree Hourly data. Forecasts are made every 6 hours, with a 1 hour time step. The values extracted are wind speed and direction at 10m altitude, and ground solar irradiance (DSWRF - Downward Short-Wave Radiation Flux), which takes into account cloud coverage. In order to obtain an estimate of those values at current time, an interpolation is made between two forecasts (the one at the beginning of the hour, and the one at the end of the hour).

Topology data

We use the Natural Earth Data Cultural Vectors country subdivisions (map admin subunits).

Contribute

Want to help? Join us on slack at http://slack.tmrow.co.

Logger

We have a public logger which shows warnings and errors for all parsers.

Running locally

To get started, clone or fork the repository, and install Docker.

The frontend will need compiling. In order to do this, open a terminal in the root directory and run

docker-compose build

One you are done, you can start the application by running

docker-compose up

This will watch over source file changes, running nonstop and watching changes you make in the code to recompile the frontend if needed. Note this only builds for the English language (which will be faster as not all languages need to be built). If you want to build all languages, change the command of the web-watch-en section of docker-compose.yml from command: npm run watch-en to command: npm run watch.

Head over to http://localhost:8000/ and you should see the map! Note that the backend is responsible for calculation carbon emissions. The map data displayed comes from a mock server providing dummy data from state file.

If you have issues building the map locally check out the Troubleshooting section below for common problems and fixes.

Once you're done doing your changes, submit a pull request to get them integrated into the production version.

Updating region capacities

If you want to update or add production capacities for a region then head over to the zones file and make any changes needed to the capacity map. Values are in MW. The zones use ISO 3166-1 codes as identifiers, a list of which can be found here.

Adding a new region

As a first step, do a search for the region on our GitHub, as contributors may have explored things before. It is very simple to add a new country. The electricityMap backend runs a list of so-called parsers every 5min. Those parsers are responsible for fetching the generation mix of a given country (check out the existing list in the parsers directory, or look at the work in progress).

I have not found a new data source You can help us by opening a new issue for the relevant region and add the relevant individual or organization to contact (ideally with their contact details on email, Twitter and phone). Doing so will ensure that we and other visitors interested can contact them to make them aware of the electricityMap. Usually, energy agencies, governments, transmission system operators are good potential sources. If you can't code, this is an amazing way to help us!

I have found a data source and I could help build a parser A parser is a python3 script that is expected to define the method fetch_production which returns the production mix at current time, in the format:

def fetch_production(zone_key='FR', session=None, target_datetime=None, logger=None):
    return {
      'zoneKey': 'FR',
      'datetime': '2017-01-01T00:00:00Z',
      'production': {
          'biomass': 0.0,
          'coal': 0.0,
          'gas': 0.0,
          'hydro': 0.0,
          'nuclear': None,
          'oil': 0.0,
          'solar': 0.0,
          'wind': 0.0,
          'geothermal': 0.0,
          'unknown': 0.0
      },
      'storage': {
          'hydro': -10.0,
      },
      'source': 'mysource.com'
    }

The session object is a python request session that you can re-use to make HTTP requests.

target_datetime is used to fetch historical data (when available). logger is a logging.Logger whose output is publicly available so that everyone can monitor correct functioning of the parsers.

The production values should never be negative. Use None, or omit the key if a specific production mode is not known. Storage values can be both positive (when storing energy) or negative (when the storage is emptied).

The parser can also return an array of objects if multiple time values can be fetched. The backend will automatically update past values properly.

Once you're done, add your parser to the zones.json and exchanges.json configuration files. Finally update the real-time sources.

Run all of the parser tests with the following command from the root directory:

python -m unittest discover parsers/test/

For more info, check out the example parser or browse existing parsers.

Generating a new map

If your changes involve altering the way countries are displayed on the map a new world.json will need to be generated. Make sure you're in the root directory then run the following command:

docker-compose run --rm web ./topogen.sh

For a more detailed explanation of how the map is generated see here.

Testing parsers locally

In order to test your parser, make sure first that you have installed the required modules as described (consider using a virtual environment) in parsers/requirements.txt: for that you can run

pip install -r parsers/requirements.txt

From the root folder, use the test_parser.py command line utility:

python test_parser.py FR price  # get latest price parser for France
python test_parser.py FR  # defaults to production if no data type is given
# test a specific datetime (parser needs to be able to fetch past datetimes)
python test_parser.py DE --target_datetime 2018-01-01T08:00

Many of the tests require API keys of the data or web service providers and will therefore fail with an error message like

Exception: No ENTSOE_TOKEN found! Please add it into secrets.env!

In such cases, please browse the website related to the provider and ask for an API key. As soon as you got hold of the API key, make it an environment variable. This fixes the error.

Update the map

We've added a testing server locally.

To add a new country to the map, run:

PYTHONPATH=. python mockserver/update_state.py <zone_name>

from the root directory, replacing <zone_name> by the zone identifier of the parser you want to test. This will fetch production and exchanges and assign it a random carbon intensity value. It should appear on the map as you refresh your local browser.

Troubleshooting

  • ERROR: for X Cannot create container for service X: Invalid bind mount spec "<path>": Invalid volume specification: '<volume spec>'. If you get this error after running docker-compose up on Windows, you should tell docker-compose to properly understand Windows paths by setting the environment variable COMPOSE_CONVERT_WINDOWS_PATHS to 0 by running setx COMPOSE_CONVERT_WINDOWS_PATHS 0. You will also need a recent version of docker-compose. We have successfully seen this fix work with v1.13.0-rc4. More info here: docker/compose#4274.

  • No website found at http://localhost:8000: This can happen if you're running Docker in a virtual machine. Find out docker's IP using docker-machine ip default, and replace localhost by your Docker IP when connecting.

About

A real-time visualisation of the CO2 emissions of electricity consumption

https://www.electricitymap.org

License:MIT License

Languages

Language:Python 58.2%Language:JavaScript 26.6%Language:HTML 10.9%Language:CSS 3.5%Language:Jupyter Notebook 0.4%Language:Shell 0.3%Language:Dockerfile 0.1%