owid / etl

A compute graph for loading and transforming OWID's data

https://docs.owid.io/projects/etl

owid/etl Issues

ETL freezing on step `data://meadow/climate/2023-12-20/surface_temperature`
Updated 3 months ago2
ETL build error due to size mismatch of snapshot file
Updated 3 months ago1
:bug: mysqlclient==2.2.4 raising `decimal.InvalidOperation`
Updated 3 months ago
🐛 chart-revision: improve handling of large datasets
Updated 3 months ago
:hammer: Replace DVC by custom code
Closed 3 months ago
Project: Wizard to update steps
Updated 3 months ago1
Improvements of `etl-datadiff`
Updated 4 months ago3
wizard: integrate fasttrack
Closed 4 months ago1
🎉 wizard: add harmonizer
Closed 4 months ago1
🎉 wizard: GPT-based tool to ask anything about metadata
Closed 4 months ago1
Improvements on `version_tracker`
Closed 4 months ago2
🐛 dummy datasets not being loaded
Updated 4 months ago1
Issues and improvements on `add_regions_to_table`
Updated 4 months ago1
metadata: display and presentation fields need to be automatically propagated on operations
Updated 4 months ago1
Increase ETL_EPOCH
Updated 4 months ago1
wizard: make apps more modular
Closed 4 months ago3
Improvements of `etl-staging-sync`
Updated 4 months ago2
Improvements of `etl-metadata-migrate`
Closed 4 months ago1
Roadmap: data engineering 2024
Updated 4 months ago
Can't delete some containers -- dataset is busy
Closed 6 months ago4
HMD STMF stopped publishing excess mortality data for most countries
Updated 4 months ago1
About 15% of indicators give 404s in the data API
Updated 4 months ago3
Data API should have offline mode when working locally
Updated 4 months ago
Apply a strict schema to YAML metadata files
Updated 4 months ago1
Origins should have an ETL path in Grapher
Updated 4 months ago2
metadata: ETL lets you define a string `description_key`, but then grapher creates a list of bullet points with just one letter
Updated 4 months ago3
enhance: deprecate `etl.data_helpers.population`
Updated 4 months ago1
metadata: Raise warnings if guidelines are not respected
Closed 4 months ago1
Warn if metadata guidelines are not fulfiled when running ETL steps
Updated 4 months ago
Snapshot config r2 issue
Closed 4 months ago3
Support data pages for our most important metrics
Closed 4 months ago3
`etl-staging-sync` fails if `--staging-created-at` is not passed
Closed 5 months ago2
Move ETL buildkite pipelines into the repo
Closed 5 months ago3
UN SDG metadata links stopped working
Closed 5 months ago2
wizard: garden step failing to load (but all others work fine)
Closed 5 months ago2
S3 metadata is out of date from the DB (`catalogPath`)
Updated 5 months ago3
Create app etl-archive-steps
Updated 5 months ago2
Warning for `display.name` but no `title_public`
Closed 5 months ago
:bug: Walden CI failing on `master` due to difficulty installing PyYAML
Closed 6 months ago
:bug: Failing garden step for tuberculosis data
Closed 6 months ago2
Add at least one dataset level tag to all datasets that have new metadata
Closed 6 months ago14
metadata: Discourage the use of `display.name` in ETL metadata
Closed 6 months ago15
:bug: KeyError when running any snapshot-private step
Closed 6 months ago
Issue with Tag table in grapher step
Closed 6 months ago3
Buildkite failure when building from scratch - SDG dataset
Closed 6 months ago
`NameError: name '_mysql' is not defined` when running ETL steps
Closed 7 months ago1
Fill spurious missing nuclear data with zeros in energy datasets
Closed 7 months ago
Add indicator's short_name to catalog path to make it unique
Closed 7 months ago5
bug: MySQLdb.IntegrityError
Closed 7 months ago3
`owid-catalog` can't read a table outside of ETL
Closed 8 months ago2