Network thingy.
Authors: Nathan Ackerman, Paul Elder
Router, UI:
- gcc
- pthreads
- libcurl (Ubuntu, Debian: libcurl4-openssl-dev, Alpine Linux: curl-dev)
auto network setup:
- bash
- coreutils
Nameserver (optional; see Setup):
- python3
- flask (
pip3 install flask) - flask_restful (
pip3 install flask_restful)
Docker-based router machines:
- docker (Ubuntu, Debian: docker.io)
Running Docker may require the user to be in the Docker group. To do this:
usermod -a -G docker <username>
As can be seen from the Dependencies section, there are four components: the router, the ui, the nameserver, and the auto network setup.
The routers themselves communicate with each other via IP address and port, but the nameserver is necessary for the network administrator to communicate to the routers (via the UI), and to initialize the network (to tell the routers which address and port its neighbours will listen on).
To run the nameserver, simply run python3 name_server.py. Make sure port
5000 and your nameserver machine is accessible from your router machines and
your machine that will run the UI.
There is a demo nameserver active at http://amanokami.net:5000 . If you do
decide to use your own nameserver, make sure to change the two #define s in
src/naming.c for NAME_SERVER_NAME and NAME_SERVER_ADD before compiling
your router and UI.
The router program can be run on either baremetal or in Docker containers (they can run in VMs, or whatever else as well, but we will not provide instructions on how to do so). If running on baremetal, keep in mind that the IP addresses that the routers use to communicate with each other must be unique and routable.
To compile the router for baremetal: make -C src (to compile with debug
flags: make debug -C src)
To compile Docker containers containing the router: make (or make debug
for compiling with debug flags)
The routers must be started up before connecting them into a network. The
syntax for running the router is (after compilation, the binary will be found
in src/router):
./src/router <router name> <router config file> [<network device name>]
Example: ./src/router jack configs/config eth0 starts a router named “jack”
with the router configuration specified in configs/config and communicating
with the network via network device eth0 (eth0 is the default).
The network device name will tell the router from which network device to get the IP address from, for example eth0, or wlan0. This IP address will then be declared to the nameserver along with its name, and this IP address is how other routers (and the UI) will communicate with the current router.
The config file is simply a list of 5 numbers, specifying, in order:
- LSA sending interval (microseconds)
- LSA generating interval (microseconds)
- Link cost sending interval (microseconds)
- Alive message sending interval (microseconds)
- initial LSA age (integer)
See configs/config for an example.
For baremetal, simply run the router program on each machine.
For Docker, see the Network initialization section. Note that the Docker
containers will only have the router configurations that are in configs/
at the time that the container is built, so if you add or change any
configs, you must build the container again (make).
The UI is how we will communicate with the routers. To compile it, run
make ui -C src. To connect to a router, run ./src/ui <router name>. The UI
binary will be in src/ui.
Note that the auto network initialization also requires the UI to be compiled.
Once the router program is running on each machine, we can now connect them into a network. We can do this manually through the UI by using the add neighbour command, or we can do it automatically via a script. The syntax for the network initialization script is:
./setup_network.sh <network config file> <flag: boot docker containers> <flag: docker containers use same router config>
Examples:
./setup_network.sh networks/simple- starts a network based on the network configurationnetworks/simple, where the routers are all currently running on baremetal (or Docker containers that are already running)./setup_network.sh networks/simple 1- boots up Docker containers for every node in the network specified innetworks/simple, and then connects the routers into a network. Note that every router in each Docker container will read its configuration fromconfigs/<router name>.config. Sincenetworks/simplehave nodes bob, alice, and eve, configuration filesconfigs/bob.config,configs/alice.config, andconfigs/eve.configmust exist at the time of building the Docker container (make)../setup_network.sh networks/simple 1 1boots up Docker containers for every node in the network specified innetworks/simple, and then connects the routers into a network. Note that all the routers in each Docker container will read its configuration fromconfigs/config, which must exist at the time of building the Docker container (make)
The last two flags are optional (they can either be omitted or set to 1), but the network config file is required.
The network config file is a simplified ini format. The section headers denote a node, and the body of each section is simply a list of neighbours. Note that if two nodes are neighbours, in the network config it only needs to be specified under one of the node’s sections. eg. if nodes A and B are neighbours, then A’s section must have B, but B’s section doesn’t need A (or vice versa).
An example network config with two nodes connected to each other:
[bob]
alice
[alice]See networks/ for more example network configurations.
If using Docker routers, you can set the first flag to 1 and the script will
spin up the Docker containers and their router programs for you, and then
initialize the network. Note that if the second flag is not set, then in the
configs directory you must have a router config file for each router, in
configs/<router name>.config. If you do set the second flag
then you only need one router config file configs/config and all Docker
routers will use the same router config.
If Docker complains that the cs2520slsrp_default network cannot be found,
then run make network and try the network setup again.
You can, of course, run Docker containers manually:
docker run --network=cs2520slsrp_default router ./router <router name> <router config file>
For both baremetal and Docker containers, it is possible to spin up new machines/routers and add them to the network after the first initialization. It is, however, not recommended to disconnect routers from the network, as Undefined Behavior will occur.
Now that you have a network set up, here are a few things that you can do (all via the UI) for any node in the network:
- add neighbours
- view neighbour list
- view routing table
- send file
For sending a file, the UI should be self explanatory. Please note though,
that the file that you are sending must be in the current working directory from
where the UI is being run, and the receiving router will put the file in the
working directory from which the router was run. Also note that you can only
send files to routers (to connect to a running Docker container, get the
container ID via docker ps or docker container ls , and then get a shell
in it via docker exec -it <container id>).
This means that, for example, if you ran ui as ./src/ui , then the file that
you want to send must be in ./ . If you ran the ui as ./ui (by first
cd’in into src), then the file must be in ./src . The location to which the
file will be received by the router will be similar (the working directory,
not the location of the executable binary).
To kill the Docker containers, run ./kill_docker.sh .
- Packet delay simulation
In src/router.c there is a #define DELAY_PROB - this is the probablity
(in percent) that any packet will be delayed. Feel free to play around with
this.
- Disconnecting a router from the network causes Undefined Behavior
- Packet error simulation
If you get a compilation error along the lines of “list.h: No such file or
directory”, then run make clean -C src .
Theoretically the network configuration file should work with defining double-sided edges, but it is recommended to define only one side for each edge.
We were unable to test on baremetal thanks to The Great Firewall of Pitt. Hopefully the Elements Cluster doesn’t have such a restriction. Docker should work.
When a router spins up, it registers its name, ip, and the listening port given by the OS to the name server. The name server is how routers in the network initially find routers to connect to. The router will then listen for connections on the listening port, spawning a new thread to handle any packet it receives, which then passes the responsibility of the packet to the thread that pertains to it.
The name server has two important functionalities. Firstly, it allows the routers to know what port to find another router listening on (reported by the ui), since this is randomly decided by the OS, but it also gives an important abstraction for the network administrators. The routers only work off of ip addresses, but the name server lets the startup and UI control work off of human readable names which the name server translates into IP addresses and ports.
To administer the router, the ui component communicates with a single router at a time through network sockets. We went with a network based UI because it enabled administering any router on the network from any machine. It is lacking the obviously needed authentication which we didn’t implmenent per it not being within the scope of this project. Once a UI instance is running, you can send the commands to the router to add a neighbour, obtain the neighbours list, obtain the routing table, and send a file.
A router that receives a UI command to add a neighbour will send a neighbour request to the desired neighbour and wait for an ack. If the ack times out it will try again. It tries a total of three times but this count may be changed. Upon neighbour acquisition, config variables are exchanged. Sending intervals will adjust to the max interval of the two neighbours.
Once a router has obtained a neighbour, the alive thread will monitor if that neighbour is still alive. If 10 (can be changed) alive messages go without an ack, the link is considered down.
While a neighbour is alive, the link cost thread will monitor the cost of the link to that neighbour. It does this by timing the rtt for a link cost message. The cost is computed as an exponentially decaying average.
LSA’s are periodically created with the cost of links to each neighbouring router. They are then sent to each of the router’s neighbours, who also forward them. They use an increasing sequence number to denote newer ones, and an age counter to account for errors in sequence numbers. When the router tries to send its or someone else’s LSA, it will wait for an ack and retry after a timeout. Only when it has forwarded the LSA to each of its neighbours successfully will it stop sending that LSA until it gets a newer one. LSAs are not sent back in the direction in which they came.
With the knowledge from all of the LSAs, a router is able to build its view of the network topology. The link state database is stored in the form of an adjacency matrix for ease of use in building the routing table. The routing table is rebuilt periodically by running Dijkstra on the LSD. Once the paths are computed, we can build a mapping from the destination router to the neighbouring router to send the packet to.
Now that the routers have a map of the network and know how to route packets within it, we are able to do file transfer. When the UI sends a file to the router, the router will split it into fragments of 1200 bytes of data plus the packet overhead. These fragments are stored at the receiving router and rebuilt and written to a file when the last fragment has arrived. When routers receive file transfer packets, they check if they are the destination; if they are not, they use the routing table to decide who to forward the packet to to reach that destination. The sending thread of the sending router will die either when it receives acks of all of its fragments it needed to send, or if a timeout occurs indicating the something is wrong and the file cannot be sent (this never actually happens).