tgel0 / spotify-data

Spotify API to BigQuery serverless pipeline

🎧 Spotify Data Project

A serverless solution for getting the audio features data from the Spotify Web API into a BigQuery database.

💻 Installation

1. Clone this repo
2. Initialize gcloud gcloud init
3. Deploy Cloud Function

gcloud functions deploy get_spotify_audio_features_data_to_bigquery --runtime python38 --trigger-topic spotify_topic --set-env-vars CID='YOUR_SPOTIFY_CID',SECRET='YOUR_SPOTIFY_SECRET',PROJECT_ID=YOUR_GCP_PROJECT_ID,DATASET_ID=YOUR_BIGQUERY_DATASET_NAME,TABLE_ID=YOUR_BIGQUERY_TABLE_NAME

4. Create a Cloud Scheduler job to run automatically (optional)

🛠️ Built With

🖥️ APIs

• Spotify Web API

🐍 Python libraries:

• Spotipy - Python wrapper for Spotify Web API

☁️ Google Cloud Platform services

• Cloud Functions
• BigQuery
• Cloud PubSub
• Cloud Scheduler

🌱 Notebooks, Datasets, Blogs

• Blog post 'Spotify Data Project Part 1 - from Data Retrieval to First Insights'
• Blog post 'Using Data to Find the Most Popular Tracks of the Summer on Spotify'
• Notebook part 1: data retrieval: github | nbviewer
• Notebook part 2: data exploration: github | nbviewer
• Dataset via Kaggle (latest update: April 2019)