These are example Terraform scripts to demonstrate how to execute Arc against a remote cluster.
For AWS:
- aws-single-master for a single instance.
- aws-cluster for a multi-instance cluster.
These both assume the default security group has access to SSH to your EC2 instances. There are sample user-data-* scripts in the ./templates which have commands for mounting the local SSD of certain instance types to /data. If used the following docker run commands should be used:
-v /data/local:/local \
-e "SPARK_LOCAL_DIRS=/local" \
-e "SPARK_WORKER_DIR=/local" \