An older code version for this tutorial can be found in the branch archive_classic_tekton.

Apply end to end security to a cloud application

The repository features a sample application that enables groups of users to upload files to a common storage pool and to provide access to those files via shareable links. The application is written in Node.js and deployed as Docker container to the IBM Cloud Kubernetes service. It leverages several security-related services and features to improve app security. It includes data encrypted with your own keys, user authentication, and security auditing.

Refer to this tutorial for instructions.

The user connects to the application.
App ID secures the application and redirects the user to the authentication page. Users can sign up from there too.
The application is running in a Kubernetes cluster from an image stored in the container registry. The image is automatically scanned for vulnerabilities.
Files uploaded by the user are stored in Cloud Object Storage.
The bucket where the files are stored is using a user-provided key to encrypt the data.
All activities related to managing the solution are logged by Cloud Activity Tracker with LogDNA.

Deploy with a toolchain and Terraform

This project comes with a partially automated toolchain capable of deploying the application to IBM Cloud. The environment including the needed services is set up using Terraform (Infrastructure as Code). The Terraform scripts are managed in a Schematics workspace.

Prerequisites

Create a standard Kubernetes cluster in a VPC (Virtual Private Cloud) with a Kubernetes version of 1.19 or higher.
Have an instance of Continuous Delivery service to be used by the toolchain.
Optionally create a specific resource group for this project

Please note that the Kubernetes cluster and the resources deployed via Terraform / Schematics have to be in the same region and resource group. The app is deployed to the Kubernetes cluster and hence in the same region and resource group, too.

Deploy resources using Terraform managed by Schematics

Either create the Schematics workspace automatically by clicking this "deploy link". Or set it up manually by going to the Schematics workspaces and using https://github.com/IBM-Cloud/secure-file-storage/tree/master/terraform as source respository including path and Terraform v0.13 as runtime.

Configure all required variables:

basename: project basename which is used as prefix for names, e.g., secure-file-storage
region is where the resources will be deployed and the location of the existing Kubernetes cluster: us-south, eu-de, ...
iks_cluster_name: name of your existing (VPC-based) Kubernetes cluster
iks_namespace: Kubernetes namespace into which to deploy the app. It will be created if it does not exist.
resource_group is the name of the IBM Cloud resource group where to deploy the services into.

Be sure to click "Save".

Next, optionally click "Generate plan" to verify everything would be working ok. Or directly click on "Apply plan" to deploy the configured resources, authorizations and service keys:

App ID
Cloud Object Storage
Cloudant NoSQL database
Key Protect

Note: By default, services are provisioned with service plans that should work in typical accounts. It means that paid plans are used. If you want to change to lite plans, you may configure different plans by changing values for variables like cos_plan, appid_plan, etc.

Deploy the app using Tekton

Click the following link to create a Tekton-based toolchain in the IBM Cloud Continuous Delivery service:

Make sure to select the region where your Continuous Delivery service is deployed.

In the dialog configure the git repository and the pipeline:

GitHub

Select the region and resource group for the toolchain.
GitHub Server: GitHub (https://github.com) - already selected
Repository type: Existing
Repository URL: https://github.com/IBM-Cloud/secure-file-storage - already selected
Repository Owner: Your GitHub user - already selected
If enabled, uncheck Enable GitHub Issues and Track deployment of code changes

Delivery Pipeline

IBM Cloud API Key: click New+ (do not click Save this key in a secrets store for reuse). The API key provides the same privileges as your user id and is used during pipeline execution
Region: Region matching the toolchain is the default, but should be adjusted to where you plan to deploy the app.
Image Registry Namespace, e.g., secure-file-storage or your username
Docker Image name: secure-file-storage default is good

Click Create.

Click Run Pipeline and choose the manual trigger to build and depploy.

Uninstall

The toolchain includes a trigger to uninstall the app. Click Run Pipeline and select that trigger. Thereafter, switch to the Schematics workspace and select the action to Destroy the resources. As an alternative, you could also select Delete. This will offer to only delete the workspace and leave the resources deployed, to delete (destroy) the resources and keep the workspace, or to delete both.

Code Structure

The file for the Infrastructure as Code, the Continuous Delivery pipeline, and the app itself are organized in several directories.

Terraform

The terraform directory holds the resource configurations files. They are loaded into the Schematics workspace to provision or destroy the resources.

Toolchain and pipeline

The directory .bluemix holds the toolchain definition including for the setup form. .tekton has the files to define the pipelines, their tasks and how the pipelines are triggered.

The file .tekton/pipelines.yaml defines two pipelines, one for the build & deploy process, one to uninstall the app. The "build & deploy" pipeline has four tasks:

git-repo-changes clones the source repository and checks for code changes to the toolchain and app.
build uses the task icr-containerize from the Open Toolchain Tekton Catalog to build the Docker image.
va-scan runs and checks vulnerability checks on the newly built image. It facilitates the IBM Cloud Container Registry for that purpose.
deploy-to-kubernetes deploys the Docker image to the Kubernetes cluster and configures the app.

Note that the pipeline definition includes an inactive check to only build and deploy the app if there were app-related changes. Uncomment that section in the source code to enable the check.

App

Located in the app directory:

File	Description
app.js	Implementation of the application.
app/credentials.template.env	To be copied to `credentials.env` and filled with credentials to access services. `credentials.env` is used when running the app locally and to create a Kubernetes secret before deploying the application to a cluster manually.
app/Dockerfile	Docker image description file.
app/secure-file-storage.template.yaml	Kubernetes deployment file with placeholders. To be copied to `secure-file-storage.yaml` and edited to match your environment.

To test locally

Follow the tutorial instructions to have the app deployed to a cluster. Specially the sections to create all the services and to populate the credentials.env file. You will need the public instead of the private COS endpoint in order to access Cloud Object Storage from your machine.
Access the tokens with https://secure-file-storage.<INGRESS_SUBDOMAIN>/api/tokens. This will shows the raw App ID authorization header together with the decode JWT tokens for your session.

In your local shell:

export TEST_AUTHORIZATION_HEADER="<value of the header attribute 'Bearer ... ...'>"

npm start

License

See License.txt for license information.

VidyasagarMSC / secure-file-storage