NeurIPS 2023 Citylearn Challenge - Starter Kit

This repository is the NeurIPS 2023 Citylearn Challenge Submission template and Starter kit! Clone the repository to compete now!

This repository contains:

Documentation on how to submit your models to the leaderboard
The procedure for best practices and information on how we evaluate your agent, etc.
Starter code for you to get started!

Competition Overview
- Competition Phases
Getting Started
How to write your own agent?
How to start participating?
Other Concepts
Important links

Competition Overview

The CityLearn Challenge 2023

Buildings are responsible for 30% of greenhouse gas emissions. At the same time buildings are taking a more active role in the power system by providing benefits to the electrical grid. As such, buildings are an unexplored opportunity to address climate change.

The CityLearn Challenge makes use of the CityLearn Gym environment as an opportunity to compete in investigating the potential of artificial intelligence (AI) and distributed control systems to tackle multiple problems within the built-environment domain. It is designed to attract a multidisciplinary participation audience including researchers, industry experts, sustainability enthusiasts and AI hobbyists as a means of crowd-sourcing solutions to these problems.

The CityLearn Challenge 2023 is a two-track challenge where either track is independent of, but may inform design choices in the other track. Both tracks make use of the same dataset of a synthetic single-family neighborhood and are run in parallel.

Competition Phases

Phase I (Warm Up Round, Aug 21 - Sep 18, 2023)

This phase provides participants with an opportunity to familiarize themselves with the competition, CityLearn environment and raise issues bordering on the problem statement, source code, dataset quality and documentation to be addressed by the organizers. A solution example will also be provided so that participants can test the submission process and see their submissions show up on the leaderboard. The submissions and leaderboard in this phase are not taken into account during Phase II and subsequent selection of winners in Phase III.

Phase II (Evaluation Round, Sep 19 - Oct 31, 2023)

This is the competition round. Participants will be able to see how each of their submissions rank against each other and how their latest submission ranks against other participants’ submissions in a public leaderboard. There are also changes made to the environment and online evaluation dataset in this phase. At the end of this phase, new submissions will be halted and existing submissions are evaluated against another different dataset but the scores and rankings are kept private and visible to only the challenge organizers.

Phase III (Review Round, Nov 1 - Nov 15, 2023)

During this phase, winners will be selected and announced. Also, the organizers will develop an executive summary of the competition that documents the preparation, winning solutions, challenges faced and lessons learned.

Getting Started

Sign up to join the competition on the AIcrowd website.
Fork this starter kit repository. You can use this link to create a fork.
Clone your forked repo and start developing your agent.
Develop your agent(s) following the template in how to write your own agent section.
Develop your reward function following the template in how to write your own reward function section.
Submit your trained models to AIcrowd Gitlab for evaluation (full instructions below). The automated evaluation setup will evaluate the submissions on the citylearn simulator and report the metrics on the leaderboard of the competition.

How to write your own agent?

We recommend that you place the code for all your agents in the agents directory (though it is not mandatory). You should implement the

register_reset - The function is used to get the prediction of the first observation after env.reset
predict - This function is called to get actions all observations, except the first.

Add your agent name in user_agent.py, this is what will be used for the evaluations.

An example are provided in agents/rbc_agent.py

How to write your own reward function?

We recommend that you place the code for all your reward functions in the rewards directory (though it is not mandatory). You should implement the

calculate - The function is used to get the reward for all observations.

Add your reward class name in user_reward.py, this is what will be used for the evaluations.

An example are provided in rewards/comfort_reward.py

How to start participating?

Setup

Add your SSH key to AIcrowd GitLab

You can add your SSH Keys to your GitLab account by going to your profile settings here. If you do not have SSH Keys, you will first need to generate one.

Fork the repository. You can use this link to create a fork.

Clone the repository

git clone git@gitlab.aicrowd.com:aicrowd/challenges/citylearn-challenge/citylearn-2023-starter-kit.git

Install competition specific dependencies!

cd citylearn-2023-starter-kit
pip install -r requirements.txt

Write your own agent as described in How to write your own agent section.
Write your own reward function as described in How to write your own reward function section.
Test your agent locally using python local_evaluation.py
Make a submission as described in How to make a submission section.

How do I specify my software runtime / dependencies?

We accept submissions with custom runtime, so you don't need to worry about which libraries or framework to pick from.

The configuration files typically include requirements.txt (pypi packages), apt.txt (apt packages) or even your own Dockerfile.

You can check detailed information about the same in the 👉 runtime.md file.

What should my code structure be like?

Please follow the example structure as it is in the starter kit for the code structure. The different files and directories have following meaning:

.
├── aicrowd.json           # Submission meta information - like your username
├── apt.txt                # Linux packages to be installed inside docker image
├── requirements.txt       # Python packages to be installed
├── local_evaluation.py    # Use this to check your agent evaluation flow locally
├── data/                  # Contains schema files for citylearn simulator (for local testing)
└── agents                 # Place your agents related code here
    ├── rbc_agent.py               # Simple rule based agent
    └── user_agent.py              # IMPORTANT: Add your agent name here
└── rewards                 # Place your reward related code here
    ├── comfort_reward.py          # Example reward function
    └── user_reward.py              # IMPORTANT: Add your reward class here

Finally, you must specify an AIcrowd submission JSON in aicrowd.json to be scored!

The aicrowd.json of each submission should contain the following content:

{
  "challenge_id": "control-track-citylearn-challenge",
  "authors": ["your-aicrowd-username"],
  "description": "(optional) description about your awesome agent",
}

This JSON is used to map your submission to the challenge - so please remember to use the correct challenge_id as specified above.

How to make a submission?

👉 submission.md

Best of Luck 🎉 🎉

Other Concepts

Evaluation Metrics

The control track score (Score_Control) is the weighted average of a thermal comfort score (Score_Control^Comfort), an emissions score (Score_Control^Emissions), a grid score (Score_Control^Grid), and a resilience score (Score_Control^Resilience). Score_Control^Grid and Score_Control^Resilience are averages of four and two KPIs respectively.

$$\textrm{Score}_{\textrm{Control}} = w_1 \cdot \textrm{Score}_{\textrm{Control}}^{\textrm{Comfort}} + w_2 \cdot \textrm{Score}_{\textrm{Control}}^{\textrm{Emissions}} + w_3 \cdot \textrm{Score}_{\textrm{Control}}^{\textrm{Grid}} + w_4 \cdot \textrm{Score}_{\textrm{Control}}^{\textrm{Resilience}}$$

$$\textrm{Score}_{\textrm{Control}}^{\textrm{Comfort}} = U$$

$$\textrm{Score}_{\textrm{Control}}^{\textrm{Emissions}} = G$$

$$\textrm{Score}_{\textrm{Control}}^{\textrm{Grid}} = \overline{R, L, P_d, P_n}$$

$$\textrm{Score}_{\textrm{Control}}^{\textrm{Resilience}} = \overline{M, S}$$

The weights are specified in the table below. In Phase I, the highest weight is given to Score_Control^Grid. By Phase II, Score_Control^Resilience is introduced and has the same weight as Score_Control^Comfort and Score_Control^Grid. Score_Control^Emissions has the lowest non-zero weight in both phases. The private leaderboard in Phase II makes use of the same weights as the public leaderboard.

Phase	$w_1$	$w_2$	$w_3$	$w_4$
Phase I	0.3	0.1	0.6	0.0
Phase II	0.3	0.1	0.3	0.3

All together, the four scores are made up of eight KPIs namely: carbon emissions (G), discomfort (U), ramping (R), 1 - load factor (L), daily peak (P_d), all-time peak (P_n) 1 - thermal resilience (M), and normalized unserved energy (S), which have been defined in the table below. G, U, M, and S are building-level KPIs that are calculated using each building's net electricity consumption (e) or temperature (T) then averaged to get the neighborhood-level value. R, L, P_d, and P_n are neighborhood-level KPIs that are calculated using the neighborhood's net electricity consumption (E). Except U, M, and S all KPIs are normalized by their baseline value where the baseline is the result from when none of the distributed energy resources (DHW storage system, battery, and heat pump) is controlled.

Name	Formula	Description
Carbon emissions	$G = \sum_{i=0}^{b-1} g^i_{\textrm{control}} \div \sum_{i=0}^{b-1} g^i_{\textrm{baseline}} \\ g = \sum_{t=0}^{n-1}{\textrm{max} \left (0,e_t \cdot B_t \right )}$	Emissions from imported electricity.
Discomfort	$U = \sum_{i=0}^{b-1} u^i_{\textrm{control}} \div b \\ u = a \div o \\ a = \sum_{t=0}^{n-1} \begin{cases} 1, \ \textrm{if} \ \lvert T_t - T_t^{\textrm{setpoint}} \rvert > c \ \textrm{and} \ O_t > 0 \\ 0 \end{cases} \\ o = \sum_{t=0}^{n-1} \begin{cases} 1, \ \textrm{if} \ O_t > 0 \\ 0 \end{cases}$	Proportion of time steps when a building is occupied, and indoor temperature falls outside a comfort band, $c$.
Ramping	$R = r_{\textrm{control}} \div r_{\textrm{baseline}} \\ r = \sum_{t=0}^{n-1} \lvert E_{t} - E_{t - 1} \rvert$	Smoothness of the neighborhood’s consumption profile where low $R$ means there is gradual increase in consumption even after self-generation is unavailable in the evening and early morning. High $R$ means abrupt change in grid load that may lead to unscheduled strain on grid infrastructure and blackouts caused by supply deficit.
1 - Load factor	$L = l_{\textrm{control}} \div l_{\textrm{baseline}} \\ l = \Bigg ( \sum_{d=0}^{n \div h} 1 - \frac{ \left ( \sum_{t=d \cdot h}^{d \cdot h + h - 1} E_{t} \right ) \div h }{ \textrm{max} \left (E_{d \cdot h}, \dots, E_{d \cdot h + h - 1} \right ) } \Bigg ) \div \Bigg ( \frac{n}{h} \Bigg)$	Average ratio of daily average and peak consumption. Load factor is the efficiency of electricity consumption and is bounded between 0 (very inefficient) and 1 (highly efficient) thus, the goal is to maximize the load factor or minimize (1 − load factor)
Daily peak	$P_d = p_{d_{\textrm{control}}} \div p_{d_{\textrm{baseline}}} \\ p_d = \Bigg ( \sum_{d=0}^{n \div h} \textrm{max} \left (E_{d \cdot h}, \dots, E_{d \cdot h + h - 1} \right )\Bigg ) \div \Bigg ( \frac{n}{h} \Bigg)$	Average, maximum consumption at any time step per day.
All-time peak	$P_n = p_{n_{\textrm{control}}} \div p_{n_{\textrm{baseline}}} \\ p_n = \textrm{max} \left (E_{0}, \dots, E_{n} \right )$	Maximum consumption at any time step.
1 - Thermal resilience	$M = \sum_{i=0}^{b-1} m^i_{\textrm{control}} \div b \\ m = a \div o \\ a = \sum_{t=0}^{n-1} \begin{cases} 1, \ \textrm{if} \ \lvert T_t - T_t^{\textrm{setpoint}} \rvert > c \ \textrm{and} \ O_t > 0 \ \textrm{and} \ F_t > 0 \\ 0 \end{cases} \\ o = \sum_{t=0}^{n-1} \begin{cases} 1, \ \textrm{if} \ O_t > 0 \ \textrm{and} \ F_t > 0 \\ 0 \end{cases}$	Same as discomfort, $U$ but only considers time steps when there is power outage.
Normalized unserved energy	$S = \sum_{i=0}^{b-1} s^i_{\textrm{control}} \div b \\ s = s^{\textrm{served}} \div s^{\textrm{expected}} \\ s^{\textrm{served}} = \sum_{t=0}^{n-1} \begin{cases} q_{n}^{\textrm{served}}, \ \textrm{if} \ F_t > 0 \\ 0 \end{cases} \\ s^{\textrm{expected}} = \sum_{t=0}^{n-1} \begin{cases} q_{n}^{\textrm{expected}}, \ \textrm{if} \ F_t > 0 \\ 0 \end{cases}$	Proportion of unmet demand due to supply shortage e.g. power outage.

Where:

$t$: Time step index;

$n$: Total number of time steps, $t$, in 1 episode;

$h$: Hours per day (24);

$d$: Day;

$i$: Building index;

$b$: Total number of buildings;

$e$: Building-level net electricity consumption (kWh);

$E$: Neighborhood-level net electricity consumption (kWh);

$A$: Electricity rate ($/kWh);

$B$: Emission rate (kg_CO₂e/kWh);

$T$: Indoor dry-bulb temperature (^oC);

$T^{\textrm{setpoint}}$: Indoor dry-bulb temperature setpoint (^oC);

$c$: Thermal comfort band ($\pm T^{\textrm{setpoint}}$);

$O$: Occupant count (people);

$F$: Power outage signal (Yes/No); and

$q$: Building-level cooling, domestic hot water and non-shiftable load energy demand (kWh).

Time and compute constraints

For Phase I, your agent should complete 1 episode in 30 minutes. Note that the number of episodes and time can change depending on the phase of the challenge. However we will try to keep the throughput requirement of your agent, so you need not worry about phase changes. We only measure the time taken by your agent. For compute, you will be provided a virutal machine with 2 CPU cores and 12 GB of RAM.

Local Evaluation

Participants can run the evaluation protocol for their agent locally with or without any constraint posed by the Challenge to benchmark their agents privately. See local_evaluation.py for details. You can change it as you like, it will not be used for the competition. You can also change the simulator schema provided under data/schemas/warm_up/schema.json, this will not be used for the competition.

Contributing

🙏 You can share your solutions or any other baselines by contributing directly to this repository by opening merge request.

Add your implemntation as agents/<your_agent>.py.
Import it in user_agent.py
Test it out using python local_evaluation.py.
Add any documentation for your approach at top of your file.
Create merge request! 🎉🎉🎉

📎 Important links

💪 Challenge Page: https://www.aicrowd.com/challenges/neurips-2023-citylearn-challenge
🗣 Discussion Forum: https://discourse.aicrowd.com/c/neurips-2023-citylearn-challenge
🏆 Leaderboard: https://www.aicrowd.com/challenges/neurips-2023-citylearn-challenge/problems/control-track-citylearn-challenge/leaderboards

Roberock / citylearn2023_kit