pvsone / temporal-money-transfer

The canonical money transfer Go application extended for use with Temporal Cloud

Temporal Money Transfer

This is a simple project for demonstrating Temporal with the Go SDK and Temporal Cloud.

This project is based on money-transfer-project-template-go and the corresponding tutorial here: https://learn.temporal.io/getting_started/go/first_program_in_go/

Basic instructions

Step 0: Temporal Cloud

This guide assumes you already have access to a Temporal Cloud account, as described in How to get started with Temporal Cloud

Step 1: Clone this Repository

In a terminal instance, clone this repo.

git clone https://github.com/pvsone/temporal-money-transfer
cd temporal-money-transfer

Step 2: Issue Certificates

The Temporal docs provide options for Issuing CA Certificates

For this guide, we will use certstrap to create a root CA and end-entity certificate.

Note: In Step 3 of this guide, we will Create a Namespace, during which we will supply a value for the Namespace Name. We will use the Namespace Name as the Common Name of the end-entity certificate that is created here in Step 2.

# set the Namespace Name for use as the Common Name in the certstrap commands below
export TEMPORAL_NAMESPACE_NAME=my-namespace

# Initialize a new certificate authority
certstrap init --common-name "My Cert Auth"

# Request a certificate, with a common name equal to the namespace name
certstrap request-cert --common-name ${TEMPORAL_NAMESPACE_NAME}

# Sign certificate request and generate the end-entity certificate
certstrap sign ${TEMPORAL_NAMESPACE_NAME} --CA "My Cert Auth"

Step 3: Create a Namespace

Follow the steps at How to create a Namespace in Temporal Cloud. In addition set the following field values:

• Set CA Certificates field value to the contents of the file out/My_Cert_Auth.crt.
• Set Certificate Filters > Filter 1 > Common Name field value to the value of $TEMPORAL_NAMESPACE_NAME defined above, e.g. my-namespace

The Certificate Filter ensures that only the end-entity certificate that we created in Step 2 can be used to access this Namespace. Without the filter in place then any certificate signed and generated by the "My Cert Auth" could be used to access this Namespace.

After successful creation, the Namespace will be given a unique Id of the form <TEMPORAL_NAMESPACE_NAME>.<TEMPORAL_ACCOUNT_ID>, e.g. my-namespace.a1bc2. Export this value as environment variable in your shell for use in commands in subsequent steps.

# replace 'a1bc2' with your account id
export TEMPORAL_ACCOUNT_ID=a1bc2
export TEMPORAL_NAMESPACE=${TEMPORAL_NAMESPACE_NAME}.${TEMPORAL_ACCOUNT_ID}

Test the Namespace Config using the Temporal CLI

Install the Temporal CLI if you have not previously done so. See Temporal CLI for instructions.

Set the following environment variables for use by the Temporal CLI

export TEMPORAL_ADDRESS=${TEMPORAL_NAMESPACE}.tmprl.cloud:7233 
export TEMPORAL_TLS_CERT=./out/${TEMPORAL_NAMESPACE_NAME}.crt
export TEMPORAL_TLS_KEY=./out/${TEMPORAL_NAMESPACE_NAME}.key

Test the connection to the Temporal Cloud service using the Temporal CLI

temporal operator namespace describe ${TEMPORAL_NAMESPACE}
# the command will return the Namespace details if successful

Step 4: Run the Workflow

# start using command line arguments
go run start/main.go \
  -address ${TEMPORAL_NAMESPACE}.tmprl.cloud:7233 \
  -namespace ${TEMPORAL_NAMESPACE} \
  -tls_cert_path ./out/${TEMPORAL_NAMESPACE_NAME}.crt \
  -tls_key_path ./out/${TEMPORAL_NAMESPACE_NAME}.key

# alternatively, if the environment variables ${TEMPORAL_ADDRESS}, ${TEMPORAL_NAMESPACE}, ${TEMPORAL_TLS_CERT}, and ${TEMPORAL_TLS_KEY} are set, then start without command line arguments
go run start/main.go

Observe that Temporal Cloud Web UI reflects the workflow, but it is still in "Running" status. This is because there is no Workflow or Activity Worker yet listening to the TRANSFER_MONEY_TASK_QUEUE task queue to process this work.

Step 5: Run the Worker

In another terminal instance, run the worker. Notice that this worker hosts both Workflow and Activity functions.

# start using command line arguments
go run worker/main.go \
  -address ${TEMPORAL_NAMESPACE}.tmprl.cloud:7233 \
  -namespace ${TEMPORAL_NAMESPACE} \
  -tls_cert_path ./out/${TEMPORAL_NAMESPACE_NAME}.crt \
  -tls_key_path ./out/${TEMPORAL_NAMESPACE_NAME}.key

# alternatively, if the environment variables ${TEMPORAL_ADDRESS}, ${TEMPORAL_NAMESPACE}, ${TEMPORAL_TLS_CERT}, and ${TEMPORAL_TLS_KEY} are set, then start without command line arguments
go run worker/main.go

Now you can see the workflow run to completion.

Deploy Worker to Kubernetes

The worker has been built and published to Docker Hub as pvsone/temporal-money-transfer-worker:1.0.0.

To deploy the worker to Kubernetes, run the following commands (Note: the environment variables ${TEMPORAL_ADDRESS}, ${TEMPORAL_NAMESPACE}, ${TEMPORAL_TLS_CERT}, and ${TEMPORAL_TLS_KEY} must be set):

Create a secret for the TLS client certificate and key

kubectl create secret generic client-credential \
  --from-file=tls.key=${TEMPORAL_TLS_KEY} \
  --from-file=tls.crt=${TEMPORAL_TLS_CERT}

There are two manifests in the manifests directory for deploying the worker to Kubernetes

• worker-deploy.yaml - Deployment for the worker - where the worker is resposible for the mTLS connection
• worker-deploy-istio.yaml - Deployment for the worker - where Istio is responsible for the mTLS connection. This requires that Istio has been installed and that the istio-injection label has been applied to the namespace where the worker will be deployed.

# deploy the worker
envsubst < manifests/worker-deploy.yaml | kubectl apply -f -

# alternatively, deploy the worker to a namespace with Istio enabled
envsubst < manifests/worker-deploy-istio.yaml | kubectl apply -f -

Start the workflow using the CLI

temporal workflow start \
  --type MoneyTransfer \
  -t TRANSFER_MONEY_TASK_QUEUE \
  -i '{"SourceAccount": "85-150","TargetAccount": "43-812","Amount": 250,"ReferenceID": "12345"}'

Attic

Old stuff that shouldn't be used, but I'm keeping around for reference.

Let's Encrypt Certificates

Warning: You probably don't want to use Let's Encrypt to create client certificates. While it will work technically, it is advisable to create your own Certificate Authority to use for issuing and managing client certificates. See https://community.letsencrypt.org/t/can-i-create-client-certificates-for-a-received-letsencrypt-certificate/78627

To use Let's Encrypt, I repeated the steps with the following modifications:

Step 2: Issue CA Certificates

Use acme.sh to generate the certs from a shell. For me, I have a personal domain (pvslab.net) registered with AWS Route53, so I issued the cert using the ACME Route53 DNS API based verification.

export AWS_ACCESS_KEY_ID=XXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXX

acme.sh --issue \
  --dns dns_aws \
  --domain sullivan-dev.tcld.pvslab.net \
  --server letsencrypt \
  --preferred-chain "ISRG"

The --server and --preferred-chain values are important to generate certs fully compatible with Temporal Cloud. I imagine there are other values that could be used here, but these are the ones that worked for me.

The important files generated from the acme.sh command are:

• Cert: Written to ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/sullivan-dev.tcld.pvslab.net.cer
• Key: Written to ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/sullivan-dev.tcld.pvslab.net.key
• Intermediate CA cert: Written to ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/ca.cer

The Intermeidate CA cert must be combined with the Root CA cert before uploading to Temporal Cloud. Get the CA for the ISRG Root from the certificates page on Let's Encrypt or using the following command:

# get the ISRG self-signed Root CA from letsencrypt
curl -O https://letsencrypt.org/certs/isrgrootx1.pem

Then combine Intermediate CA with the Root CA in a single PEM file

# combine the Intermediate CA with the ISRG Root CA
cat ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/ca.cer isrgrootx1.pem > cacerts.pem

Step 3: Update the Namespace

Update the Namespace configuration with the following field values:

• Set CA Certificates field value to the contents of the file cacerts.pem.
• Set Certificate Filters > Filter 1 > Common Name field value to sullivan-dev.tcld.pvslab.net

Step 4: Run the Workflow

go run start/main.go \
  -address ${TEMPORAL_NAMESPACE}.tmprl.cloud:7233 \
  -namespace ${TEMPORAL_NAMESPACE} \
  -tls_cert_path ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/sullivan-dev.tcld.pvslab.net.cer \
  -tls_key_path ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/sullivan-dev.tcld.pvslab.net.key

Step 5: Run the Worker

go run worker/main.go \
  -address ${TEMPORAL_NAMESPACE}.tmprl.cloud:7233 \
  -namespace ${TEMPORAL_NAMESPACE} \
  -tls_cert_path ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/sullivan-dev.tcld.pvslab.net.cer \
  -tls_key_path ~/.acme.sh/sullivan-dev.tcld.pvslab.net_ecc/sullivan-dev.tcld.pvslab.net.key