jeffbrennan / TexasPandemics

Scrapes, cleans, and analyzes Texas COVID-19 data

UTHealth COVID-19 Dashboard

Scraping, statistics, and diagnostics of Texas COVID-19 data

Installation

1. Clone the repository using git clone --depth 1 https://github.com/jeffbrennan/TexasPandemics.git. The --depth 1 flag is optional and will only pull the latest commit to increase install speed.
2. Install the requirements using pip install -r requirements.txt (preferably in a virtual environment)
3. Configure the environment variables in .env (see .env.example for an example)
4. Configure the dagster environment variables see the dagster docs
5. Launch dagster in the cli using dagster dev in the root directory.
6. Navigate to localhost:3000 in your browser to view the dagster dashboard.
7. Create a new run using Materialize All

Future Work

• Dockerize the project
• Use a database to store the data instead of csv files
• Pull in additional sources
• Add a more robust testing suite & run diagnostics

Current ETL: