Stockholm 2018-10-03
Curate your Playlist by Data Engineering.
You will need Java 8, Scala and SBT. For more details, please take a look at this in-depth guide.
This hack will be using your personal listening history from your Spotify account through our Web API. To be able to use this data, you need to create a token that gives you permission to read it.
To generate a token, go to the Spotify developer portal and press Get Token.
Make sure you select the following scopes for this exercise:
- playlist-modify-public
- playlist-modify-private
- user-top-read
- playlist-read-private
You can read more about what the different scopes provide in this guide.
Copy and paste this token into spotify-codepub/src/main/resources/spotifyToken and it will applied to any API calls you make.
When you have downloaded the project you will need to compile it. This process will do things like the Avro magic we mentioned. To do this, navigate to the project folder in your terminal and run:
sbt compile
Open spotify-codepub/src/main/scala/com.spotify.data/example/CodePubPlaylistJob.
We have created some boilerplate code for you, with comments to explain what is going on and some steps for you to complete in order to get this pipeline working.
Don't hesitate to ask your peers or the organizers if you get stuck.
If you are using an IDE like IntelliJ, you can simply run the main method of the pipeline class to see it in action.
If you are unable to do this, you can run your pipeline using sbt. In this case, navigate to the project folder in your terminal and simply run:
sbt
Now you are in a Scala environment. Since the codebase consists of one project encompassing two smaller sub-projects, we want to navigate to the one with the pipeline logic. We do this by running:
project spotify-codepub
From here, we can run the main method of our class with this command:
runMain com.spotify.data.example.CodePubPlaylistJob