Repository with resources used for preparation to Databricks Certified Associate Developer - Apache Spark (Pyspark)
A good way to understand how Apache Spark works, is by deploying it locally on your machine through Docker. You can even leverage Docker Compose to create virtual workers that could replicate a Spark cluster. In thi session we are going to describe how you could do it.
The first step is to install Docker and Docker Compose. You can find tutorials on the official documentations for each project:
- Install Docker
- [Install Docker Compose](- Install Docker ) To make sure the installations were successful, try running the commands on your terminal.
docker --version
docker-compose --version
Next, we need to create a docker-compose.yaml
file with all the configurations needed to a local deploy.