SvetlanaM / data-migration

Data cleaning, transformations and external API integrations in Pandas. Converting data from CSV to PostgreSQL database

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Pandas data transformation scripts

Scripts for data migrations from static csv sheets to the PostreSQL database.

Main purpose of these scripts:

  1. Data cleaning
  2. Data transformation
  3. Normalisation and denormalisation
  4. Creating static tables
  5. Foreign Key and Many to Many Key connections

There are also a lot external integrations like:

  1. Download current bank courses for currency conversion from US Dollar to any currency
  2. Get currency code from currency name
  3. Transform images and upload them to the Digital Ocean Spaces
  4. Get the GPS coordinates from the name
  5. Categorise regions, countries and continents
  6. Generate usernames and passwords
  7. Optimise images

About

Data cleaning, transformations and external API integrations in Pandas. Converting data from CSV to PostgreSQL database


Languages

Language:Jupyter Notebook 100.0%