A curated list of open data sources to analyze and compare cities in a holistic way à la data science and empower citizens.
- Cost of living
- Salary
- Businesses
- Transportation
- Energy
- Health
- Water
- Waste/recycling and sanitation
- Internet
- Heavy industry
- Agriculture
- Population
- Workforce
- Education
- Crime
- Immigration/emigration
- Climate
- Air quality
- Soil quality
- Noise
- Waterways
- Parks/forests/Conservation areas
- Natural disasters
- Pollution
- Residential planned/new/stock
- Commercial planned/new/stock
- Real estate market
- Zoning and Land use
- Energy demand
- Happiness
- Culture
- Communities
This list is for anyone looking to get a quantitative perspective on a specific place. This is not a database, it is a list of data sources. Based on the geographically distributed nature of this data and my personal interests the availability of data is going to be biased. It is my intention that you add data sources from your geographic areas of interest via pull requests to grow open data usage and improve diversity and detail. Once a significant amount of data sources exist for several places good analysis and comparisons can be made.
The idea is to curate an alphabetical list of data sources by geographic level from macro to micro scale, in descending subsets, following as closely as possible:
- Nation
- state/province/canton
- city/metropolitan area
- district/neighbourhood
| Level | Topics | Status | Description | Format | Source | link |
|---|
Example of Levels column:
| Level | meaning |
|---|---|
| * | datasets at nation level for multiple nations |
| Switzerland/* | datasets at canton level for multiple cantons in Switzerland |
| */* | datasets at state/province/canton level for state/province/canton cantons in multiple nations |
| United States/Illinois/Chicago/ | datasets at city level for Chicago, Illinois |
| United States/Illinois/*/ | datasets at city level for multiple cities in Illinois |
While including so many levels many seem confusing it allows for greater specification, and with the increasing socio-economic links between a city and its metropolitan area this specification is worthwhile.
The main distinction is whether there is a provided API, raw data downloads, (in .csv/.xml/.json/etc.), or aggregate stats. The aggregate stats are calculated on data that is not made available directly.
Many data sources are themselves data services/bases/warehouses operated by governments or institutions. These sources are listed in a similar fashion to stand-alone data sets, but contain multiple topics and levels. These sources are also more likely to provide an API. Individual data sets of particular interest within these collections can be listed separately to make best usage of this list.
| Level | Topics | Status | Description | Format | Source |
|---|---|---|---|---|---|
| * | Health | up | Disease burden, Health | raw | Institute for Health Metrics and Evaluation (IHME) |
| * / * / * / * | Economics | up | Cost of living user contributed data about cities | aggregate stats (paid API) | Numbeo |
| Africa / * / * / * | * | up | Independent data from African continent | raw | openAfrica |
| European Union / * / * / * | * | up | EU government open data | API and raw | EU Open Data Portal |
| Canada / */ * | Development | up | National Bank House Price Index | aggregate stats | Teranet-National Bank |
| Canada / * / * / * | * | up | Canadian Open Data Inventory | raw | Open Government Canada |
| Canada / Alberta / Edmonton / * | * | up | Edmonton City data | raw | City of Edmonton Open Data |
| Canada / Ontario / Brampton / * | * | up | Brampton City data | raw | OpenGov Brampton |
| Canada / Ontario / Toronto / * | * | up | Toronto City data | raw | City of Toronto Open Data |
| Canada / Ontario / Toronto / * | Development | up | Cleared building permits | raw | City of Toronto Open Data |
| Canada / Ontario / Toronto / * | Development | up | Active building permits | raw | City of Toronto Open Data |
| Canada / Ontario / Toronto / * | Development | up | 3D building massing | raw | City of Toronto Open Data |
| Canada / Ontario / Waterloo / * | * | up | Waterloo City data | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Infrastructure | up | Trails pedestrian and cyclist counts | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Infrastructure | up | Traffic Closures (1/16/2017-7/10/2018) | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Infrastructure | up | Road Traffic Volumes | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Infrastructure | up | Sidewalks | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Infrastructure | up | Bridges | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Development | up | Buildings | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Development | up | Community Gardens | raw | City of Waterloo Open Data |
| Canada / Ontario / Waterloo / * | Development | up | Libraries | raw | City of Waterloo Open Data |
| Canada / Ontario / York Region / * | * | up | York Region data | raw | York Region Open Data |
| Germany / ? / ? / | Development | up | energy consumption of 107 municipal buildings | raw | https://im.iism.kit.edu/sciber.php |
| Switzerland / Zürich / Zürich / * | * | up | City of Zürich open data | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Development | up | Social housing construction | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Development | up | Construction activities since 2009 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Development | up | Completed and demolished apartments since 2009 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Development | up | Completed and demolished apartments since 2009 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Development | up | Residences by property type and age group since 2008 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Development | up | Cinema locations 1907-2018 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Environment | up | Air quality 1983-2012 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Infrastructure | up | City energy consumption 1990-2016 | raw | Stadt Zürich |
| Switzerland / Zürich / Zürich / * | Infrastructure | up | City primary energy balance 1990-2016 | raw | Stadt Zürich |
| United States / * / * / * | * | up | US Federal government open data | API and raw | Data.gov |
| United States / * / * / * | Development | up | MicroSoft US Building Footprints | raw | Links on Github Microsoft/USBuildingFootprints |
| United States / Massachusetts / Cambridge / * | * | up | City of Cambridge data | raw | www.cambridgema.gov |
| United States / Massachusetts / Cambridge / * | Development | up | Cambridge property database | raw | www.cambridgema.gov |
- http://www.pewinternet.org/datasets/
- https://www.openicpsr.org/openicpsr/search/studies
- https://datahub.io/search
- https://github.com/awesomedata/awesome-public-datasets
- https://github.com/datasciencemasters/data
- https://www.reddit.com/r/datasets/
- https://github.com/CityofToronto/vz_challenge
- https://www.icpsr.umich.edu/icpsrweb/ICPSR/search/studies
- https://github.com/City-Bureau/city-scrapers
- https://github.com/citygram/citygram-services
- http://dataportals.org/
- https://www.openicpsr.org/openicpsr/search/studies
- https://registry.opendata.aws/
- http://www.economagic.com/
To the extent possible under law, Thomas Stesco has waived all copyright and related or neighboring rights to this work.