Datasets

This repo contains information about how to obtain or compile a dataset for the course's project problem.

Geospatial Datasets

Geospatial datasets contain map locations of regions or points of interest.

1. Compile your dataset

You can compile your own dataset using Overpass QL language that runs on Overpass turbo. You can use this query language to mine OpenStreetMaps data, filter it, and get it ready to be used by osmnx or any library that parses .osm files. Below is a quick review about using Overpass API, which is the official API for reading data from OpenStreetMap servers. All the online routing services and software use it. Additionally, we will usually use Nominatim to do geocoding/geo-decoding; translating addresses to/from (latitude-longitude).

Also be aware of the fact that most of the time if you are building a dataset over a very big area in the map, the graph parsed from the data by osmnx won't be complete, even though there are physically feasible routes that could make the graph complete and connect all the nodes. This deficiency is usually because of the incomplete relations and data of osm -- don't worry about that now.

For these cases, we will be using OSRM to fill these gaps when needed, with the help of some utilities in utilities/src/poi.py. This can be seen in some of the case studies.

Using `OverPass QL`

Fire up Overpass turbo and run these scripts and export it as .osm files.

Finding the bounding box around an area of interest is a recurring problem in writing OverPass QL queries. To solve for that, we can use bbox finder. Don't forget to change the coordinate format to latitude/longitude at the right corner after drawing the polygon around the area of interest.

Using `Overpass turbo`'s Wizard

Overpass turbo's Wizard provides an easy way to auto-generate Overpass QL queries. Wizard syntax is similar to that of a search engine. An example of Wizard syntax is amenity=hospital that generates an Overpass QL query to find all the hospitals in a certain region of interest. Hospital locations will be visualized on the map and can be downloaded/copied using the "Export" button. The data can be exported as GeoJSON, GPX, KML, raw OSM data, or raw data directly from Overpass API. You can then use osmnx to read .osm files with osmnx.graph_from_xml.

Some examples of Overpass turbo's wizard syntax include:

amenity=restaurant in "Toronto, Canada" to find all resturants in City of Toronto.
amenity=cafe and name="Tim Hortons" to find all Tim Hortons coffee shops.
(amenity=hospital or amenity=school) and (type:way) to find hospitals and schools with close ways mapped.
amenity=hospital and name~"General Hospital" will find all hospitals with "General Hospital" part of their names.

Contributing: Do you have something cool and want to share it with us? If you collected your data by downloading it directly from OpenStreetMaps and did some filtering with osmfilter, open a pull request with the details of the data and how you filtered it. If you collected your data with overpass turbo, please attach your Overpass QL script with the data so we can replicate your results, and maybe we can learn a thing or two from your script.

2. Available open datasets

3. Commercially available datasets

Traffic Datasets

Parking Tickets Datasets

Public Transport Networks Datasets

Google Transit APIs
...

Planned events, road work, and other temporary changes to the road network

Traffic Crashes Datasets

Emission Datasets

Environment Datasets

Mobility-aware urban design, active transportation modeling and access analysis for amenities and public transport

Urbano
...

Accessibility

Wheelmap
...

Crime map

Crime map
...

Bus Routes

Bus routes
...

Open Source Projects

GIScience
...

About

A home for miscellaneous data.

Apache License 2.0