ParallelCluster Manager - Make HPC Easy

This project is a front-end for AWS Parallel Cluster

Quickly and easily create HPC cluster in AWS using ParallelCluster Manager. This UI uses the AWS ParallelCluster 3.0 API to Create, Update and Delete Clusters as well as access, view logs, and build Amazon Machine Images (AMI's).

Want to request a new feature?

First checkout the Roadmap
If you don't already see your feature, open a feature request

You can get started with your first cluster in as little as 15 minutes using the links below.

Quickstart (15 mins) 🚀

Launch the stack in your AWS account by clicking on one of the below regions:

Region	Launch
Ohio (us-east-2)
North Virginia (us-east-1)
Ireland (eu-west-1)
Frankfurt (eu-central-1)

More Regions (Click to expand)

Region	Launch
Oregon (us-west-2)
California (us-west-1)
London (eu-west-2)
Paris (eu-north-1)
Stockholm (eu-north-1)
Middle East (me-south-1)
South America (sa-east-1)
Canada (ca-central-1)
Hong Kong (ap-east-1)
Tokyo (ap-northeast-1)
Seoul (ap-northeast-2)
Mumbai (ap-south-1)
Singapore (ap-southeast-1)
Sydney (ap-southeast-2)
GovCloud West (us-gov-west-1)

Enter your email and wait (~15 mins) for the stack to go into CREATE_COMPLETE. Using the code from your email login to the Web UI at the address specified by the PclusterManagerUrl in the Outputs tab of the main pcluster-manager stack.

For more details see the Getting Started Guide.

Screen Shot

System Architecture

Costs

ParallelCluster Manager is built on a serverless architecture and falls into the free tier for most uses. I've detailed the dependency services and their free-tier limits below:

Service	Free Tier
Cognito	50,000 Monthly Active Users
API Gateway	1M Rest API Calls
Lambda	1M free requests / month & 400,000 GB-seconds of compute / month
Image Builder	No-Cost except EC2
EC2	~15 mins one-time to build Container Image

Typical usage will likely cost < $1 / month.

Updating

To update the the latest version, run the following, make sure to set the region to where you deployed the stack:

git clone https://github.com/aws-samples/pcluster-manager.git
cd pcluster-manager/
./scripts/update.sh --region us-east-1  # should be region where stack is deployed

Local Development

To run AWS ParallelCluster Manager locally, start by setting the following environment variables:

export AWS_ACCESS_KEY_ID=[...]
export AWS_SECRET_ACCESS_KEY=[...]
export AWS_DEFAULT_REGION=us-east-2
export API_BASE_URL=https://[API_ID].execute-api.us-east-2.amazonaws.com/prod  # get this from ParallelClusterApi stack outputs
export ENV=dev

Install dependencies by running:

pip3 install -r requirements.txt

Backend with Cognito

From the Cognito service page of the AWS account where pcluster-manager has been deployed, click on the user pool and then on the App Integration tab. In the App client list at the bottom, make note of the Client ID, then click on the App client and click Edit in the Hosted UI section, adding http://localhost:5001/login to the Allowed callback URLs.

Then export the following variables:

export SECRET_ID=<the value of the UserPoolClientSecretName output from the PclusterManagerCognito stack>
export SITE_URL=http://localhost:5001
export AUDIENCE=<the value of the Client ID noted in the previous step>
export AUTH_PATH=<the UserPoolAuthDomain output of the ParallelClusterCognito nested stack>

Start the API backend by running:

./scripts/run_flask.sh

Start the React frontend by running:

cd frontend/
npm install # if this is your first time starting the frontend
npm run dev

Lastly, navigate to http://localhost:5001

Frontend

Disable authentication (HMR does not work when working on http://localhost:5001), note that this has to be exported in shell where the backend runs:

export ENABLE_AUTH=false

Start the API backend by running:

./scripts/run_flask.sh

Start the React frontend by running:

cd frontend/
npm install # if this is your first time starting the frontend
npm run dev

Navigate to http://localhost:3000

Typescript

The project has been converted to Typescript using ts-migrate(an in depth explanation can be found here). The tool automatically adds comments similar to // @ts-expect-error when typing errors cannot be fixed automatically: if you fix a type error either by adding a missing third party declaration or tweaking the signature of a function, you can adjust automatically the codebase and remove //@ts-ignore comments using npm run ts-reignore.

Testing

Launch tests of the API backend by running:

pytest

For detailed information on how to invoke pytest, see this resource.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

stephenmsachs / pcluster-manager