lvdh / foldingathome-on-aws

Run the Folding@home client on AWS.

Folding@home on AWS

• Folding@home on AWS
  • Prerequisites
    • Workstation
    • AWS
  • Usage
    • CLI Commands
    • Configuration
    • Manage CloudFormation Stacks
      • Create
      • Update
      • Status
      • Delete
  • Implementation
    • Deployment Tool
    • Directory Structure
    • Linting

Prerequisites

Workstation

Local or remote system with access to the AWS API.

• CLI environment (developed on Linux, should be portable to Mac and WSL)
• Python >=3.7
• pip3 >=20.0.2
• pipenv >=2018.11.26
  • pip3 install --user pipenv==2018.11.26

Note: pipenv will install additional runtimes and dependencies in a dedicated, virtual Python environment.

AWS

• IAM User/Role
  • With sufficient permissions to manage the AWS resources defined by the CloudFormation templates.
• AWS Access Keys, temporary or long-term, exported to environment variables
  • Best Practices for Managing AWS Access Keys
  • Environment Variables To Configure the AWS CLI
  • Use of aws-vault is recommended.

Usage

CLI Commands

# Show all available make targets
make

Configuration

1. Accept the terms and conditions of the NVIDIA AMI

   1. Log into the AWS console.
   2. Go to the NVIDIA Deep Learning AMI product in the AWS Marketplace.
   3. Click Continue to Subscribe.
   4. Read through the terms. If you agree, click Accept Terms.

2. Review the main Sceptre configuration:

   1. project_code

      Project code, eg. myproject.

   2. region

      Configure the AWS Region to deploy to.

      Note: Sceptre configurations may be cascaded for multi-Region deployments.

3. Review and update the CloudFormation Parameters and configuration.

Manage CloudFormation Stacks

Create

# Deploy all stacks
make stack=fah launch

# Deploy the network stack
make stack=fah/network launch

Note: Sceptre figures out dependencies between stacks at runtime.

Update

Note: Additional to creating non-existing stacks, launch applies pending updates (if any) to stacks which already exists.

# Update all
make stack=fah launch

# Update the network stack
make stack=fah/network launch

Status

# Get status of all stacks
make stack=fah status

# Get status of the network stack and its dependencies
make stack=fah/network status

Delete

Destroy all AWS resources of one or many stacks:

Note: Stacks which depend on the targeted stacks will be deleted as well.

# Delete the network stack, as well as stacks which depend on it.
make stack=fah/network delete

Implementation

Deployment Tool

This repository implements the Sceptre tool to facilitate CloudFormation deployments.

Tip: To learn how to use the config/ and templates/ subdirectories, read up on Sceptre's online resources:

• Sceptre documentation
• Sceptre GitHub

Directory Structure

For context, see Sceptre Directory Structure.

cfn/                      # Sceptre root
├── config/               # CloudFormation Parameter values, Tags
│   ├── config.yaml       # AWS Profile, AWS Region, Project Code
│   └── fah/              # StackGroup root
│       ├── network.yaml  # Configuration for eg. VPC, gateways, ...
│       └── ...
└── templates/            # CloudFormation templates
    └── fah/              # StackGroup root
│       ├── network.yaml  # Template for eg. VPC, gateways, ...
│       └── ...

Linting

Sceptre resolves dependencies between stacks at runtime. If required, it will update stacks on which the targeted stack relies.

As a result, the lint step does not know beforehand which stacks will receive updates. For this reason, the lint step lints all CloudFormation templates and Lambda payloads regardless of the targeted stack.