This repository contains code that can be used to A) Collect historical data using the OSHDB API, and B) Clean, filter, analyze, and visualize this data to understand characteristics of data production and maintenance for a given area.
This code requires an OSHDB extract for a given area of interest. Extracts can be downloaded directly from HeiGIt's download server, or generated using the ETL process described here.
This code also requires a simple CSV file with spatial and temporal parameters that describe the extent of your area of interest. The format of this CSV file should follow the example provided here, case_studies.csv. As is included in this file, you should specify a brief string identifier for your case study or area of interest (case-study-name).
- As is described in this first steps tutorial, you should begin by creating a new Java maven project and add OSHDB as a dependency.
- Using the OSHDB API, the StreamOSHDB.java file will generate a CSV file that contains all NODES or WAYS that were produced in OSM within the spatial and temporal extents defined in your case_studies.csv file. This file requires the bounding box, start date, end date, and output file path variables to be defined. This file should be run multiple times if multiple case studies or areas are being investigated. An example output file can be found within the Data folder. Note that in this file, all OSM user IDs have been replaced with the number, 1.
- Note that, to be compatible with the subsequent data analysis script, this output CSV file should follow the case-study-name-nodes.csv/case-study-name-ways.csv naming convention.
- This output CSV file can then be processed using the Data_Processing notebook. This notebook contains a series of functions for loading and cleaning data, calculating summary statistics, parsing feature tags, creating shapefiles, and calculating maintenance. This notebook also contains code for creating numerous data visualizations to communicate key results. This Data_Processing file requires you to specify your case-study-name as an input parameter to read in the appropriate data. This processing script also assumes that you have two .csv files output from Step #2 - one for OSM nodes and one for OSM ways.