ArcticLampyrid / travellings-graph

This project is to build a graph of links between Travellings members, and analyze the connections between them.

Travellings Graph

Have you ever heard of Six Degrees of Separation? Six degrees of separation is the idea that all people are six or fewer social connections away from each other. As a result, a chain of "friend of a friend" statements can be made to connect any two people in a maximum of six steps.

This project is to build a graph of links between Travellings members, and analyze the connections between them, showing how many steps in average are needed to connect to/by each member.

Usage

docker buildx build -t travellings-graph .
mkdir -p data
docker run --rm -v $(pwd)/data:/app/data travellings-graph crawl
docker run --rm -v $(pwd)/data:/app/data travellings-graph analyze
ls -l data

# Serve as API server
docker run --rm -v $(pwd)/data:/app/data -p 8471:8471 travellings-graph serve

Crawl

You can run with subcommand crawl to crawl the data from the source.

Warning

Because there are no standard format for exchanging Links, the data is crawled with many tricks, and may not be accurate. If you find any error, please let me know.

The member list is from Travellings List, and saved in data/members.json.
The Links data is crawled from each member's Links page, and saved in data/friends.lines.json.

Analyze

You can run with subcommand analyze to analyze the data.

During the analysis, the graph is firstly built with NetworkX, and saved in data/graph.gexf. All nodes are labeled with their Member ID in Travellings List.

Tip

You can use Gephi to visualize the graph and analyze the connections. For Arch Linux, you can install Gephi with pacman -S gephi.

Also, a basic analysis is generated and saved in data/analysis.csv, as well as a simple report in data/analysis.md. The results include the average steps needed to connect to/by each member.

Serve

You can run with subcommand serve to serve as an API server. The server is built with FastAPI, and you can access the API document at /docs or /redoc endpoint.

Results

A copy of the completed data was shared on my blog (view it). Note that the data may be outdated, and the results may be different from the latest.

License

Licensed under the MIT License.