Slack Lunch Club

Live Demo

Overview

Read the series of blog posts for more information about this project.

This application allows a user to input the following:

slack teams via "Login with Slack"
availability
location

To receive a weekly match email with another user who:

shares a common slack team
shares an available day to meet for lunch
is nearby
has not been previously matched

This is a useful tool to help connect slack team members in a easy, fun, and spontaneous way.

Technology

Core Tech:

React ( create-react-app )
GraphQL
Serverless Framework ( AWS provider )
ArangoDB ( graph database )

Supporting Tech:

styled components
flow type
jest
puppeteer
codecov.io
papertrail
sentry
webpack
prettier
eslint
docker + docker-compose

AWS Services:

S3
EC2
APIGateway
Lambda
CloudFormation
CloudWatch
CloudFront
Route53
VPC
IAM
Certificate Manager
CodeBuild

The end result is a first-class developer experience, which aims to provide a complete production-ready SaaS stack. This stack is primarily for small teams, startups, indie-developers who are on a budget, but also want technology that can scale with increased traction.

Setup

Part 1 - Third Party Services

create a slack developer account
create a mailgun account
create a papertrail account
create an AWS account
create a codecov account
create a sentry account
register a custom domain name ( https://domains.google or namecheap.com recommended )
download and install Docker

Part 2 - Download dependencies

run npm run dev to download and start the docker containers
create a new GUUID here and keep it in a SAFE place. This one string is a single point of failure for your application secrets.
run export ENCRYPTION_KEY=<YOUR_NEW_GUUID>
run npm run install:npm

Part 3 - Set environment variables

run cp ./secrets.template.js ./secrets.js
get AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY here, make sure it has admin credentials
get CODECOV_TOKEN from codecov.io, after adding this repo, go to the repo settings, and look for the "Repository Upload Token"
get GITHUB_TOKEN here, grant it "repo" access

set these variables to their correct values in ./secrets.js

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
CODECOV_TOKEN
GITHUB_TOKEN

run cp ./global-aws/secrets.template.js ./global-aws/secrets.js
GITHUB_REPO is the git repo where this file lives
HOSTED_ZONE_NAME this is the custom domain name you registered in Part 1
both PAPERTRAIL_HOST and PAPERTRAIL_PORT can be found here
set VPN_USER and VPN_PASSWORD to whatever you like

set these variables to their correct values in ./global-aws/secrets.js

GITHUB_REPO
HOSTED_ZONE_NAME
PAPERTRAIL_HOST:
PAPERTRAIL_PORT
VPN_USER
VPN_PASSWORD

run npm run encrypt && cd global-aws && npm run encrypt && cd .. to generate your encrypted secrets files

Part 4- Deploy global AWS resources

run exit then npm run dev to reload bash shell
run export ENCRYPTION_KEY=<YOUR_NEW_GUUID> again
you must accept the Terms of Use for the AWS Marketplace OpenVPN EC2 AMI here: https://aws.amazon.com/marketplace/server/procurement?productId=fe8020db-5343-4c43-9e65-5ed4a825c931
create a non-public S3 bucket with the name found in ./global-aws/serverless.yml > provider.deploymentBucket
run cd global-aws && npm run deploy && cd ..
wait about 2 minutes for AWS to provision resources, check the status here. The deployment will not be able to complete until we configure the DNS in the next Part.
create a papertrail alert with Query = error -("errors") here and have it forward to your email

Part 5 - Configure DNS

navigate here and click on the public hosted zone for your custom domain
copy the 4 name servers from the NS record, and paste them into your domain registrar's settings. ( at domains.google.com for example )
navigate here and create a new domain ( same name as the AWS public hosted zone )
verify your mailgun domain, by adding the 4 DNS records listed under Domain Verification & DNS, these records must be added to the AWS public hosted zone
to receive email at the public hosted zone ( ie, admin@yourdomain.com ) we must create a mailgun route to forward the email to an actual email server ( ie, gmail.com ) by navigating here and creating a new route like so:
```
Expression Type: Match Recipient
Recipient: ".*@YOUR-DOMAIN.com"
Actions: forward your_name@gmail.com
```
wait 10-20 minutes for DNS changes to propagate
click apps.mailgun.com > Domain Verification & DNS > Check DNS Records Now ( keep clicking this until all are green and you receive a confirmation email )
once the domain is confirmed, we need to resend the SSL certificate requests so that AWS CloudFormation can resume stack creation. Navigate here, select the certificate, then click Actions > Resend validation email. Do this for both certificates.
after receiving the SSL confirmation email, open it and click "Approve"
wait another 5 minutes for the global-aws stack to finish deploying

Step 6 - Configure ArangoDB databases

Navigate here, select the global-aws stack, and click the URL under Outputs > OpenVPNServerURL
enter the VPN_USER and VPN_PASSWORD you chose in Part 3, download the OpenVPN client app for your OS and install it
use the same IP, username, and password to connect to the VPN via the OpenVPN client ( trust the profile when prompted )
once the VPN connection is established, navigate to the URL: http://ARANGO_DB_IP:8529, where ARANGO_DB_IP is defined by the CloudFormation Outputs > ArangoDBInstanceId
log in to the ArangoDB _system database with username "root" and password defined by CloudFormation Outputs > ArangoDBInstanceId

add 2 new databases via the UI:

Databases > Add Database > "YOUR-DOMAIN-staging"
Databases > Add Database > "YOUR-DOMAIN-production"

add 2 database users via the UI:

Users > Add User > username="YOUR-DOMAIN-staging",password="GUUID_1"
Users > Add User > username="YOUR-DOMAIN-production",password="GUUID_2"

NOTE: The passwords should be 2 different GUUIDs generated here like the ENCRYPTION_KEY in Part 2. Make a note of these passwords, we will need them in the next step.
edit both the users' permissions so that they only have "Administrate" to their associated database, "No Access" to other databases, and "Administrate" to "_system" database.
disconnect from the VPN

Part 7 - Deploy the backend API

run cp ./backend/secrets.template.js backend/secrets.js
SLACK_CLIENT_ID and SLACK_CLIENT_SECRET can be created here under Settings > Basic Information > App Credentials. Create 2 sets of ID/SECRET pairs, one for staging and one for production.
AUTH_TOKEN_SECRET should be a another new GUUID (different for each stage)
DB_BACKUP_BUCKET_NAME is defined by the CloudFormation Outputs > DBBackupBucket
SMTP_PASS can be found at app.mailgun.com > domains > YOUR_DOMAIN > Domain Information > Default Password

set these variables in ./backend/secrets.js to their correct values:

SLACK_CLIENT_ID
SLACK_CLIENT_SECRET
AUTH_TOKEN_SECRET
DB_USER:
DB_PASS:
DB_NAME
DB_BACKUP_BUCKET_NAME
SMTP_PASS

change the "domainName" variable to YOUR-DOMAIN
run cd backend && npm run encrypt to generate you encrypted secrets file
create a non-public S3 bucket with the name defined by ./backend/serverless.yml > provider.deploymentBucket
run npm run deploy:staging && npm run deploy:production && cd ..

add the following Redirect URLs to the Slack App's "OAuth & Permissions" here:

https://localhost:4000/auth/slack/return
https://api.YOUR-DOMAIN/auth/slack/return
https://api-staging.YOUR-DOMAIN/auth/slack/return

Part 8 - Deploy the front end

update these 3 files to include the correct value for REACT_APP_BACKEND_API_URL, REACT_APP_SENTRY_URL, and REACT_APP_GOOGLE_ANALYTICS_ID:
```
.env.development
.env.staging
.env.production
```
REACT_APP_BACKEND_API_URL is the same as defined by ./backend/secrets.js > DOMAIN_NAME, but with the protocol prepended ( ie. https://api-staging.YOUR-DOMAIN.com )
REACT_APP_SENTRY_URL can be found here
REACT_APP_GOOGLE_ANALYTICS_ID can be found here (make sure to create a view & filter for staging and production)
create a non-public S3 bucket with the name defined by ./frontend/serverless.yml > provider.deploymentBucket
update the npm commands in ./frontend/package.json deploy:staging and deploy:production to have the correct domainName argument
run cd frontend && npm run deploy:staging && npm run deploy:production && cd ..
it will take about 30 minutes to deploy, as AWS must provision a CloudFront distribution
after the deploys complete, the app should be live and working at your custom domain!

Part 9 - Setup and test CI/CD pipeline

link your github.com account to AWS. Under Build Projects > Edit (your project) > Source: What to build you must authenticate github via OAuth
add the ENCRYPTION_KEY as an AWS SSM variable here via Create Parameter > Name="/YOUR-DOMAIN/ENCRYPTION_KEY",Type="Secure String", Value="YOUR_KEY"
edit ./buildspec.yml to use the correct value at env.parameter-store.ENCRYPTION_KEY
push the 3 encrypted secrets files and 3 modified .env files to a new branch and create a PR into master. That PR should show you the build status on CodeBuild. After the build succeeds, merge the PR, and delete the branch. Another build will be kicked off. This time, after it succeeds, your code will be deployed to production. (Note: any build that is not from a push to a PR branch or master with fail, resulting in false positive failures. This should be fixed in the future)

Part 10 - Add SSL certificates to OS keychain

if you don't do this, the react app cannot hot reload on file save
run npm run start this will start the backend and frontend

The following 2 certificates must be added to the keychain locate here:

frontend/node_modules/webpack-dev-server/ssl/server.pem

backend/config/ssl/cert.pem

Read here for OSX instructions, and here for other platforms

Conclusion

If you have followed the previous steps, you should now have your own web application deployed, along with aggregated logs, monitoring, error reporting, CI/CD, and code/type coverage - congratulations!
This AWS stack should be very affordable (ideally under $10 per month), here are the monthly estimates:
```
$2.50 - Route53
$2.00 - CodeBuild
$8.00 - EC2
```
These prices assume you are Free Tier eligible. After your 12 months run out you can get free credits by joining AWS Activate
You can also get $5000 worth of credits by through your https://producthunt.com/ship membership ( $60 per month )
There are many other ways of getting AWS Credits if you search online, though ideally if your service has traction you can charge customers to pay the AWS bill

How to upgrade ArangoDB ( non-major versions )

in the application repo, replace all instances of old version with new version ( ie. 3.3.3 -> 3.3.8 )
you can ssh into the EC2 instance and run this command: sudo apt-get update; sudo apt-get upgrade
for major version upgrades, be very careful migrating the data

Generate a new ArangoDB EC2 AMI & instance

in the application repo, replace all instances of old version with new version ( ie. 3.3.3 -> 3.3.8 )

run these commands (OSX):

brew install packer
git clone https://github.com/arangodb-helper/ami-appliance.git
cd ami-appliance/AMAZON
export AWS_ACCESS_KEY_ID=YOUR_ID
export AWS_SECRET_ACCESS_KEY=YOUR_KEY

replace all in ami-appliance project: "t3.medium" > "t2.nano"

run commands:

packer build --var 'arangodb_version=3.3.8' --var 'arangodb_name=arangodb3' --var 'arangodb_repo=arangodb33' template.json

wait about 5 minutes for packer to finish
copy the ami id, paste it into ./global-aws/resources.yml:ArangoDBInstance.Properties.ImageId, and run cd ./global-aws && npm run deploy to create a new EC2 instance
once the new EC2 instance is running, copy its private IP address, paste and update the arangodb.YOURDOMAIN.COM A record in all hosted zones
after the new db instance is deployed, you have to VPN into the web UI, create the databases and users, then restore the data with the db-restore lambda function. Be very careful.

Optional ( recommended ) extras

enable the cost explorer and set a budget to protect against surprise expenses: https://console.aws.amazon.com/billing/home?#/costexplorer
use this tool to help keep your code high quality: https://www.codefactor.io
integrate github.com with slack via https://slack.github.com
use this free service to monitor the website: https://uptimerobot.com
monitor client errors (nice sentry integration): https://logrocket.com
you can protect the master branch so that no PRs can be merged unless the build succeeds: https://github.com/mikestaub/slack-lunch-club/settings/branches/master

Contributing

This project was my first time using almost all of these technologies. I'm sure there are many things that can be improved. If you think of something do not hesitate to open an issue! I am also looking for help expanding the application so please let me know if you want to collaborate, everyone is welcome. :)

mikestaub / slack-lunch-club