How to add a worker node from a different server?

[klazaj]

I successfully ran the python template on my local machine. Now I want to add more workers, located in different machines. Is that possible to achieve within this template?

[Gezim Sejdiu]

Hi @klazaj ,

great, good to hear that. Yes, that is possible to add more worker nodes to your cluster. There are many ways how to do it. Via k8s, docker swarm, or even plan docker setup deployed separately on each node on the host network.

I have in plan to provide a simple guideline on how to deploy Spark on Swarm and maybe you can also re-use it for your purpose.

Best regards,

[klazaj]

@GezimSejdiu I just had to change the version to 3, and initiate a swarm with many workers. It works well for me.