maxmouchet / internet-visualization

Source code of "Towards an interactive visualization of the Internet".

internet-visualization

This repository contains the source code for the web interface demoed in "Towards an interactive visualization of the Internet" (slides, dataset).

Reproduce the web interface

To reproduce the web interface demoed in the talk, download the tileset (5.2GB) and start the server:

curl -Lo data/iris-2022-04-02.mbtiles 'https://zenodo.org/record/8034723/files/iris-2022-04-02.mbtiles?download=1'
docker compose up

Then, open http://localhost:1234 in your browser.

Rebuild the dataset

We provide four data files:

• iris-2022-04-02.edges: the IP-level graph in the LGL format
• iris-2022-04-02.layout: the coordinates of each IP address as computed by the LGL algorithm
• iris-2022-04-02.geojsonl: points and lines representing IP addresses and links, augmented with RIR and AS information
• iris-2022-04-02.mbtiles: the tileset used by Mapbox GL

In this section we show how to build the tileset starting from the IP-level graph.

If you want to run this on your own data, simply replace iris-2022-04-02.edges with your own file. You can use the write_lgl function of networkxtra to convert a graph to the LGL format.

Requirements

• minilgl for computing the layout
• tippecanoe for building the tilesets
• Python and Poetry

Download the graph

curl -Lo data/iris-2022-04-02.edges.zst 'https://zenodo.org/record/8034723/files/iris-2022-04-02.edges.zst?download=1'
zstd -d data/iris-2022-04-02.edges.zst

Compute the layout

lglayout2d -t 4 data/iris-2022-04-02.edges
mv lgl.out data/iris-2022-04-02.layout

With this graph containing 1.3M nodes and 3.5M edges this should take ~2 hours, depending on your machine and the number of threads set with -t.

Build the GeoJSON file

Create a virtual environment for the internet_maps module and enter a shell inside it:

poetry -C python/ install
poetry -C python/ shell

Augment the graph with RIR and AS information, convert the node positions to the approriate coordinate system and output a GeoJSONL file:

internet-maps geojson --bgp-date 2022-04-02 --scale 10 \
    data/iris-2022-04-02.edges data/iris-2022-04-02.layout data/iris-2022-04-02.geojsonl

This is relatively fast, although downloading the BGP RIB can take some time.

Build the tileset

We encode some metadata in the tileset name as a JSON string.

tippecanoe \
    --drop-densest-as-needed \
    --extend-zooms-if-still-dropping \
    --force \
    --hilbert \
    --maximum-zoom=g \
    --read-parallel \
    --name='{"data_source": "Iris", "vantage_point": "LIP6, Paris, France", "date": "2022-04-02"}' \
    --output data/iris-2022-04-02.mbtiles data/iris-2022-04-02.geojsonl 

This should take ~15 minutes depending on your machine.