ZiruiOu / DOTE

Original DOTE codebase.

Prerequisites

Installation instructions:

1. Install the python packages which are listed in requirements.txt.

pip install -r requirements.txt

2. Install Gurobi Solver version 9.1.2.

3. Set up the following environment variables:

• GRB_LICENSE_FILE - the path to the Gurobi license file.
• PYTHONPATH - add paths to DOTE/networking_envs and DOTE/openai_baselines.

4. In DOTE/networking_envs/networking_env/utils/shared_consts.py, set the BASE_PATH_LOCAL variable to point to the DOTE/networking_envs folder.

Data Format

Example data format can be found in DOTE/networking_envs/data/TEST.
When adding a new topology and demand matrices, create a new folder with the name of the topology in the DOTE/networking_envs/data/ folder.
The topology folder should have the following structure:

• train folder
• test folder
• latent folder
• TOPOLOGY_int.txt file
• TOPOLOGY_int.pickle.nnet file
• (optional) tunnels.txt file

The train/test folders should contain the demand matrices in the following format:
Each DM is flattened to a line, with demands separated by spaces. Each file should contain the DMs of a single week. Separate the train and test data into the test/train folders. In addition, for DOTE and RL, these folders should contain the optimal maximum-link-utilization of each DM (.opt files).
The latent folder allows for a latent representation of the data, i.e. a different input to represent the DMs. The format there is the same, only that the length of the 'DM' is the latent space dimension. This folder should also have train/test folders with the appropriate data.

TOPOLOGY_int.txt holds the topology in an adjacency matrix format.
TOPOLOGY_int.pickle.nnet file holds the links of the topology, where every line is src,dst,capacity.
tunnels.txt is an optional file, which lists all the tunnels in the topology (these tunnels can also be generated by the code, see the --paths_from flag). Each line is of the format: src dst:tunnel_01_edge_01-tunnel_01_edge_02-...-tunnel_01_edge_n,tunnel_02_ede_01-tunnel_02_edge_02-...-tunnel_02_edge_k

Obtaining the public datasets

Topology Zoo

The script DOTE/networking_envs/data/gml_to_dote.py converts topologies from GML format to DOTE's format, and generates synthetic demand matrices.
Publicly available topologies in GML format can be found at The Internet Topology Zoo.
In order to convert a topology to DOTE's format and generate synthetic DMs:\

1. In DOTE/networking_envs/data/gml_to_dote.py, set the src_dir, dest_dir and network_name variables.

2. Run gml_to_dote.py

3. Copy the generated folder to DOTE/networking_envs/data

GEANT

To run experiments on a topology similar to the one we used in the evaluation:

1. Download GEANT's demand matrices from SNDlib.
2. Extract the demand matrices
3. In DOTE/networking_envs/data/gen_geant_topo.py, set the demand_matrices_dir variable.
4. Run gen_geant_topo.py
5. Copy the generated folder to DOTE/networking_envs/data
6. To compute the optimum for the demand matrices, go to DOTE/networking_envs/data/GEANT and run DOTE/networking_envs/data/compute_opts.py

Running DOTE

Run DOTE/dote.py with the following flags:

• --ecmp_topo TOPOLOGY, sets the current topology.
• --paths_from [sp, ksp], sets the method to find tunnels (mandatory, unless a tunnels.txt file already exists).
• --so_mode [train, test]
• --so_epochs [n], sets the number of epochs
• --so_batch_size [n], sets the batch size
• --opt_function [MAXUTIL, MAXFLOW, MAXCONC], for maximum link utilization, maximum flow, and maximum concurrent flow.

Running Demand Prediction

Run DOTE/networking_envs/ml/sl_algos/evaluate.py with the following flags:

• --ecmp_topo TOPOLOGY, sets the current topology.
• --paths_from [sp, ksp], sets the method to find tunnels (mandatory, unless a tunnels.txt file already exists).
• --sl_model_type linear_regression, the type of predictor.
• --sl_type [stats_comm, eval], stats_comm to train the predictor, eval to evaluate the prediction.
• --opt_function [MAXUTIL, MAXFLOW, MAXCONC], for maximum link utilization, maximum flow, and maximum concurrent flow.

Running RL

Install python version 3.6 and the packages which are listed in requirements_rl.txt.

Run DOTE/openai_baselines/baselines/ppo2/run.py with the following flags:

• --ecmp_topo TOPOLOGY, sets the current topology.
• --paths_from [sp, ksp], sets the method to find tunnels (mandatory, unless a tunnels.txt file already exists).
• --history_action_type w_eps, sets the model to output weight per edge.
• --rl_mode [train, test]
• --num_epochs, adjusts the number of timesteps of the training.