Giters

breadchris / retool-onpremise

Deploying Retool On Prem

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Retool logo
Build internal tools, remarkably fast.


Deploying Retool on-premise

Deploying Retool on-premise ensures that all access to internal data is managed within your own cloud environment. You also have the flexibility to control how Retool is setup within your infrastructure, configure logging, and enable custom SAML SSO using providers like Okta and Active Directory.

Table of contents

Select a Retool version number

We recommend you set your Retool deployment to a specific version of Retool (that is, a specific semver version number in the format X.Y.Z, instead of a tag name). This will help prevent unexpected behavior in your Retool instances. When you are ready to upgrade Retool, you can bump the version number to the specific new version you want.

To help you select a version, see our guide on Retool Release Versions.

Simple Deployments

Get set up in 15 minutes by deploying Retool on a single machine.

Deploying on EC2

Spin up a new EC2 instance. If using AWS, use the following steps:

  1. Click Launch Instance from the EC2 dashboard.
  2. Click Select for an instance of Ubuntu 16.04 or higher.
  3. Select an instance type of at least t3.medium and click Next.
  4. Ensure you select the VPC that also includes the databases / API’s you will want to connect to and click Next.
  5. Increase the storage size to 60 GB or higher and click Next.
  6. Optionally add some Tags (e.g. app = retool) and click Next. This makes it easier to find if you have a lot of instances.
  7. Set the network security groups for ports 80, 443, 22 and 3000, with sources set to 0.0.0.0/0 and ::/0, and click Review and Launch. We need to open ports 80 (http) and 443 (https) so you can connect to the server from a browser, as well as port 22 (ssh) so that you can ssh into the instance to configure it and run Retool. By default on a vanilla EC2, Retool will run on port 3000.
  8. On the Review Instance Launch screen, click Launch to start your instance.
  9. If you're connecting to internal databases, whitelist the VPS's IP address in your database.
  10. From your command line tool, SSH into your EC2 instance.
  11. Run the command git clone https://github.com/tryretool/retool-onpremise.git.
  12. Run the command cd retool-onpremise to enter the cloned repository's directory.
  13. Edit the Dockerfile to set the version of Retool you want to install. To do this, replace X.Y.Z in FROM tryretool/backend:X.Y.Z with your desired version. See Select a Retool version number to help you choose a version.
  14. Run ./install.sh to install Docker and Docker Compose.
  15. In your docker.env (this file is only created after running ./install.sh) add the following: 
    # License key granted to you by Retool
LICENSE_KEY=YOUR_LICENSE_KEY

# This is necessary if you plan on logging in before setting up https
COOKIE_INSECURE=true
  16. Run docker-compose up -d to start the Retool server.
  17. Run sudo docker-compose ps to make sure all the containers are up and running.
  18. Navigate to your server's IP address in a web browser. Retool should now be running on port 3000.
  19. Click Sign Up, since we're starting from a clean slate. The first user to into an instance becomes the administrator.

Deploying to AWS using Opta

Just use the Deploy to AWS button below!

Deploy

Deploying on Heroku

Just use the Deploy to Heroku button below! You'll then have to go to Settings and set the LICENSE_KEY to your license key and PGSSLMODE to require.

Deploy

Manually setting up Retool on Heroku

Alternatively, you may follow the following steps to deploy to Heroku

  1. Install the Heroku CLI, and login. Documentation for this can be found here: https://devcenter.heroku.com/articles/getting-started-with-nodejs#set-up
  2. Clone this repo git clone https://github.com/tryretool/retool-onpremise
  3. Change the working directory to the newly cloned repository cd ./retool-onpremise
  4. Create a new Heroku app with the stack set to container with heroku create your-app-name --stack=container
  5. Add a free database: heroku addons:create heroku-postgresql:hobby-dev
  6. In the Settings page of your Heroku app, add the following environment variables:
    1. NODE_ENV - set to production
    2. HEROKU_HOSTED set to true
    3. JWT_SECRET - set to a long secure random string used to sign JSON Web Tokens
    4. ENCRYPTION_KEY - a long secure random string used to encrypt database credentials
    5. LICENSE_KEY - your Retool license key
    6. PGSSLMODE - set to require
  7. Push the code: git push heroku master

To lockdown the version of Retool used, just edit the first line under ./heroku/Dockerfile to:

FROM tryretool/backend:X.Y.Z

Running Retool using Aptible

  1. Add your public SSH key to your Aptible account through the Aptible dashboard
  2. Install the Aptible CLI, and login. Documentation for this can be found here: https://www.aptible.com/documentation/deploy/cli.html
  3. Clone this repo git clone https://github.com/tryretool/retool-onpremise
  4. Change the working directory to the newly cloned repository cd ./retool-onpremise
  5. Edit the Dockerfile to set the version of Retool you want to install. To do this, replace X.Y.Z in FROM tryretool/backend:X.Y.Z with your desired version. See Select a Retool version number to help you choose a version.
  6. Create a new Aptible app with aptible apps:create your-app-name
  7. Add a database: aptible db:create your-database-name --type postgresql
  8. Set your config variables (your database connection string will be in your Aptible Dashboard and you can parse out the individual values by following these instructions). Be sure to rename EXPIRED-LICENSE-KEY-TRIAL to the license key provided to you. 
    aptible config:set --app your-app-name \
    POSTGRES_DB=your-db \
    POSTGRES_HOST=your-db-host \
    POSTGRES_USER=your-user \
    POSTGRES_PASSWORD=your-db-password \
    POSTGRES_PORT=your-db-port \
    POSTGRES_SSL_ENABLED=true \
    FORCE_SSL=true \
    NODE_ENV=production \
    JWT_SECRET=$(cat /dev/urandom | base64 | head -c 256) \
    ENCRYPTION_KEY=$(cat /dev/urandom | base64 | head -c 64) \
    LICENSE_KEY=EXPIRED-LICENSE-KEY-TRIAL
  9. Set your git remote which you can find in the Aptible dashboard: git remote add aptible your-git-url
  10. Push the code: git push aptible master
  11. Create a default Aptible endpoint
  12. Navigate to your endpoint and sign up as a new user in your Retool instance

Deploying to Render

Just use the Deploy to Render button below! Here are some docs on deploying Retool with Render.

Deploy to Render

Managed deployments

Deploy Retool on a managed service. We've provided some starter template files for Cloudformation setups (ECS + Fargate), Kubernetes, and Helm.

Deploying on ECS

We provide a template file for you to get started deploying on ECS.

  1. In the ECS Dashboard, click Create Cluster
  2. Select EC2 Linux + Networking as the cluster template.
  3. In your instance configuration, enter the following:
    • Select On-demand instance
    • Select t2.medium as the instance type (or your desired instance size)
    • Choose how many instances you want to spin up
    • (Optional) Add key pair
    • Choose your existing VPC (or create a new one)
    • (Optional) Add tags
    • Enable CloudWatch container insights
  4. Select the VPC in which you’d like to launch the ECS cluster; make sure that you select a public subnet.
  5. Download the retool.yaml file, and add your license key and other relevant variables.
  6. Go to the AWS Cloudformation dashboard, and click Create Stack with new resources → Upload a template file. Upload your edited retool.yaml file.
  7. Then, enter the following parameters:
    • Cluster: the name of the ECS cluster you created earlier
    • DesiredCount: 2
    • Environment: staging
    • Force: false
    • Image: tryretool/backend:X.Y.Z (But replace X.Y.Z with your desired version. See Select a Retool version number to help you choose a version.)
    • MaximumPercent: 250
    • MinimumPercent: 50
    • SubnetId: Select 2 subnets in your VPC - make sure these subnets are public (have an internet gateway in their route table)
    • VPC ID: select the VPC you want to use
  8. Click through to create the stack; this could take up to 15 minutes; you can monitor the progress of the stack being created in the Events tab in Cloudformation
  9. After everything is complete, you should see all the resources with a CREATE_COMPLETE status.
  10. In the Outputs section within the CloudFormation dashboard, you should be able to find the ALB DNS URL. This is where Retool should be running.
  11. The backend tries to guess your domain to create invite links, but with a load balancer in front of Retool you'll need to set the BASE_DOMAIN environment variable to your fully qualified domain (i.e. https://retool.company.com). Docs here.

OOM issues

If running into OOM issues (especially on larger instance sizes with >4 vCPUs)

  • Verify the issue by going into the ECS console and checking the Service Metrics. Ideally
    • Memory utilization should fall around 40% (20% - 60%)
    • CPU utilization should be close to zero (0% - 5%)
  • If the values fall outside these ranges, increase the CPU and memory allocation in retool.yml

Deploying on ECS with Fargate

We provide Fargate template files supporting public and private subnets.

  1. In the ECS Dashboard, click Create Cluster
  2. In Step 1: Select a cluster template, select Networking Only (Powered by AWS Fargate) as the cluster template.
  3. In Step 2: Configure cluster, be sure to enable CloudWatch Container Insights. This will help us monitor logs and the health of our deployment through CloudWatch.
  4. Download the public or private template file, and add your license key and other relevant variables.
  5. Go to the AWS Cloudformation dashboard, and click Create Stack with new resources → Upload a template file. Upload your edited .yaml file.
  6. Enter the following parameters:
    • Cluster: the name of the ECS cluster you created earlier
    • DesiredCount: 2
    • Environment: staging
    • Force: false
    • Image: tryretool/backend:X.Y.Z (But replace X.Y.Z with your desired version. See Select a Retool version number to help you choose a version.)
    • MaximumPercent: 250
    • MinimumPercent: 50
    • SubnetId: Select 2 subnets in your VPC - make sure these subnets are public (have an internet gateway in their route table)
    • VPC ID: select the VPC you want to use
  7. Click through to create the stack; this could take up to 15 minutes; you can monitor the progress of the stack being created in the Events tab in Cloudformation
  8. In the Outputs section, you should be able to find the ALB DNS URL.
  9. Currently the load balancer is listening on port 3000; to make it available on port 80 we have to go to the EC2 dashboard → Load Balancers → Listeners and click Edit to to change the port to 80.
    • If you get an error that your security group does not allow traffic on this listener port, you must add an inbound rule allowing HTTP on port 80.
  10. In the Outputs section within the CloudFormation dashboard, you should be able to find the ALB DNS URL. This is where Retool should be running.
  11. The backend tries to guess your domain to create invite links, but with a load balancer in front of Retool you'll need to set the BASE_DOMAIN environment variable to your fully qualified domain (i.e. https://retool.company.com). Docs here.

Deploying on Kubernetes

  1. Navigate into the kubernetes directory
  2. Edit the retool-container.yaml and retool-jobs-runner.yaml files to set the version of Retool you want to install. To do this, replace X.Y.Z in image: tryretool/backend:X.Y.Z with your desired version. See Select a Retool version number to help you choose a version.
  3. Copy the retool-secrets.template.yaml file to retool-secrets.yaml and inside the {{ ... }} sections, replace with a suitable base64 encoded string.
    1. To base64 encode your license key, run echo -n <license key> | base64 in the command line. Be sure to add the -n character, as it removes the trailing newline character from the encoding.
    2. If you do not wish to add google authentication, replace the templates with an empty string.
    3. You will need a license key in order to proceed.
  4. Run kubectl apply -f ./retool-secrets.yaml
  5. Run kubectl apply -f ./retool-postgres.yaml
  6. Run kubectl apply -f ./retool-container.yaml
  7. Run kubectl apply -f ./retool-jobs-runner.yaml

For ease of use, this will create a postgres container with a persistent volume for the storage of Retool data. We recommend that you use a managed database service like RDS as a long-term solution. The application will be exposed on a public ip address on port 3000 - we leave it to the user to handle DNS and SSL.

Please note that by default Retool is configured to use Secure Cookies - that means that you will be unable to login unless https has been correctly setup.

To force Retool to send the auth cookies over HTTP, please set the COOKIE_INSECURE environment variable to 'true' in ./retool-container.yaml. Do this by adding the following two lines to the env section.

        - name: COOKIE_INSECURE
          value: 'true'

Then, to update the running deployment, run $ kubectl apply -f ./retool-container.yaml

Deploying on Kubernetes with Helm

See https://github.com/tryretool/retool-helm for full Helm chart documentation and instructions.

Additional features

For details on additional features like SAML SSO, gRPC, custom certs, and more, visit our docs.

Environment Variables

You can set environment variables to enable custom functionality like managing secrets, customizing logs, and much more. For a list of all environment variables visit our docs.

Health check endpoint

Retool also has a health check endpoint that you can set up to monitor liveliness of Retool. You can configure your probe to make a GET request to /api/checkHealth.

Troubleshooting

  • On Kubernetes, I get the error SequelizeConnectionError: password authentication failed for user "..."
    • Make sure that the secrets that you encoded in base64 don't have trailing whitespace! You can use kubectl exec printenv to help debug this issue.
    • Run echo -n <license key> | base64 in the command line. The -n character removes the trailing newline character from the encoding.
  • I can't seem to login? I keep getting redirected to the login page after signing in.
    • If you have not enabled SSL yet, you will need to add the line COOKIE_INSECURE=true to your docker.env file / environment configuration so that the authentication cookies can be sent over http. Make sure to run sudo docker-compose up -d after modifying the docker.env file.
  • TypeError: Cannot read property 'licenseVerification' of null or TypeError: Cannot read property 'name' of null
    • There is an issue with your license key. Double check that the license key is correct and that it has no trailing whitespaces.

Updating Retool

The latest Retool releases can be pulled from Docker Hub. When you run an on-premise instance of Retool, you’ll need to pull an updated image in order to get new features and fixes.

See more information on our different release channels and recommended update strategies in our documentation.

Docker Compose deployments

Update the version number in the first line of your Dockerfile.

FROM tryretool/backend:X.Y.Z

Then run the included update script ./update_retool.sh from this directory.

Kubernetes deployments

To update Retool on Kubernetes, you can use the following command, replacing X.Y.Z with the version number or named tag that you’d like to update to.

kubectl set image deploy/api api=tryretool/backend:X.Y.Z

Heroku deployments

To update a Heroku deployment that was created with the button above, you may first set up a git repo to push to Heroku

$ heroku login
$ git clone https://github.com/tryretool/retool-onpremise
$ cd retool-onpremise
$ heroku git:remote -a YOUR_HEROKU_APP_NAME

To update Retool (this will automatically fetch the latest version of Retool)

$ git commit --allow-empty -m 'Redeploying'
$ git push heroku master

Releases

Releases notes can be found at https://updates.retool.com.

Docker cheatsheet

Below is a cheatsheet for useful Docker commands. Note that you may need to prefix them with sudo.

Command Description
docker-compose up -d Builds, (re)creates, starts, and attaches to containers for a service. -dallows containers to run in background (detached).
docker-compose down Stops and remove containers and networks
docker-compose stop Stops containers, but does not remove them and their networks
docker ps -a Display all Docker containers
docker-compose ps -a Display all containers related to images declared in the docker-compose file.
docker logs -f <container_name> Stream container logs to stdout
docker exec -it <container_name> psql -U <postgres_user> -W <postgres_password> <postgres_db> Runs psql inside a container
docker kill $(docker ps -q) Kills all running containers
docker rm $(docker ps -a -q) Removes all containers and networks
docker rmi -f $(docker images -q) Removes (and un-tags) all images from the host
docker volume rm $(docker volume ls -q) Removes all volumes and completely wipes any persisted data

About

Deploying Retool On Prem

Languages

Language:Shell 72.3%Language:JavaScript 26.3%Language:Dockerfile 1.4%