Analyzing public lands and US Congress using graphs. Inspired by https://github.com/acannistra/landwatch
One of the most powerful features of graph databases like Neo4j is the ability to combine datasets and query across them. This project combines data about US public lands into a single graph database for analysis using graph data science and graph visualization.
Currently supports the following data sources:
-
Protected areas - USGS Protected Areas Database
-
Bills, Legislators, Committees from US Congress - Propublica Congress API
-
???
This project uses Poetry for managing Python dependencies and virtual environments.
To use the notebooks:
poetry install
poetry shell
jupyter notebook
First, download the USGS Protected Areas Database. NOTE: You may need to refresh download link here.
wget -O data/region9.zip https://prod-is-usgs-sb-prod-content.s3.amazonaws.com/622256afd34ee0c6b38b6bb7/PADUS3_0_Region_9_GeoJSON.zip\?AWSAccessKeyId\=AKIAI7K4IX6D4QLARINA\&Expires\=1672066494\&Signature\=955c4BEVO7XGRLQlSGyLaWIwqqM%3D
Extract zip file:
unzip data/region9.zip -d data/region9
Next, reproject to WGS84 using ogr2ogr:
ogr2ogr -s_srs ESRI:102039 -t_srs EPSG:4326 -f geojson data/region9/output.json data/region9/PADUS3_0Fee_Region9.json
Set root of LOAD CSV to data directory in neo4j.conf
server.directories.import=/Users/lyonwj/github/johnymontana/land-graph/data
Also, increase Neo4j maximum memory heap size in neo4j.conf:
server.memory.heap.max_size=16G
Enable file import with APOC in conf/apoc.conf:
apoc.import.file.enabled=true
// TODO: prepare database for import by creating constraints
CREATE CONSTRAINT FOR (p:Parcel) REQUIRE p.name IS UNIQUE;
CREATE CONSTRAINT FOR (g:Geometry) REQUIRE g.FID IS UNIQUE;
Import in Neo4j using apoc.load.csv:
CALL apoc.load.json('file:///region9/output.json')
YIELD value
UNWIND value.features AS feature
WITH feature WHERE feature.geometry.type = "Polygon"
MERGE (p:Parcel {name: feature.properties.Unit_Nm})
ON CREATE SET
p += feature.properties
MERGE (g:Geometry {FID: p.FID})
ON CREATE SET
g.coordinates = [coord IN feature.geometry.coordinates[0] | point({latitude: coord[1], longitude: coord[0]})]
MERGE (p)-[:HAS_GEOMETRY]->(g)
SET g:PolygonNext, to load US Congressional data using the ProPublica API see notebooks/import.ipynb
