nicklasfrahm / showcases

A repository containing showcases for my programming and architecture skills.

Showcases

A repository containing showcases for my programming and architecture skills.

TLDR

Create a .env file as shown below:

BROKER_URI=nats://nats:4222
SENDGRID_API_KEY=xxxxxxxxxxxxxxxx
SENDGRID_HTTP_URI=https://api.sendgrid.com/v3/mail/send
SPARKPOST_API_KEY=xxxxxxxxxxxxxxxx
SPARKPOST_HTTP_URI=https://api.eu.sparkpost.com/api/v1
AUTHORIZED_CREDENTIALS=user1:pass1,user2:pass2
MAIL_FROM=[e-mail]

Afterwards run:

make up

Navigate to http://localtest.me:8080 and enjoy.

Approach

I decided to create an interface in Go that exposes a .Send() function. This function is used by looping over all available providers. This implements automatic failover. In the future this could be improved by also assigning priorities. Another nice addition would be to show the external service status, which would greatly improve observability.

NOTE: There is an intentional bug (it's not a bug, it's a feature) that will cause SendGrid to fail. The application will instead use SparkPost to send the email.

Make sure to authenticate via the Basic authentication scheme as configured in the AUTHORIZED_CREDENTIALS file of your .env file. Afterwards, test the application by performing the requests in the following order:

• GET /v1/services/mail/providers
• POST /v1/mails
• GET /v1/services/mail/providers

Endpoints

Send email - POST /v1/mails

Request

{
  "recipients": ["[e-mail]"],
  "subject": "Yo! Check this out!",
  "message": "This is sent via a microservice architecture."
}

Response

{
  "recipients": ["[e-mail]"],
  "subject": "Yo! Check this out!",
  "message": "This is sent via a microservice architecture.",
  "mail_provider": {
    "name": "sparkpost-http",
    "transport": "HTTP",
    "disabled": false
  }
}

List mail providers and their status - GET /v1/services/mail/providers

Response

[
  {
    "name": "sendgrid-http",
    "transport": "HTTP",
    "disabled": false
  },
  {
    "name": "sparkpost-http",
    "transport": "HTTP",
    "disabled": false
  }
]

Architecture

The diagram below is the end-to-end architecture of the application showing the functional architecture building blocks and their connections.

Below you may find the proposed tech stacks for the different services.

Service	Technologies
Web Interface	React, Material UI, Static Hosting
Broker / Queue	NATS
HTTP API	Go, Fiber, NATS
WS API	Go, Fiber, NATS
AuhtX	Go, casbin, Redis, JWT, NATS
Users	Go, PostgreSQL, NATS
Likes	Go, Redis, NATS
Posts	Go, PostgreSQL, NATS
Conversations	Go, MongoDB, NATS
Sessions	Go, Redis, NATS
Mail	Go, SendGrid, NATS

Future improvements

Below you may find a list of future improvements:

• Automated testing with a focus of integration testing
• Abstraction of broker and gateways protocols to allow for easy migration to other technologies, such as GCP Pub/Sub

Development

This section describes the required dependencies for developing the components of the project.

This repository assumes that you develop on a debian-based Linux, such as Ubuntu 20.04 Desktop or the Windows subsystem for Linux.

Frontend

The frontend is written in JavaScript using Svelte, a fast and modern UI framework. You will need to have NodeJS installed to develop and build the application. We recommend that you use the node version manager to install NodeJS.

Afterwards navigate to the web/ folder (e.g. cd web/) and run npm ci to install the project dependencies.

Microservices

The microservices are all written in Go, because it provides the following advantages:

• Fast compilation to a small binary executable
• Ability to create statically compiled and self-contained binaries
• No need for a runtime
• Easy cross-compilation for cross-platform support
• Strong typing to help with consistent and stable interfaces
• Wide industry adoption via projects like Kubernetes or Docker
• Easy implementation of concurrency via goroutines
• Convenient error handling via simple ifstatements
• Superior performance and memory footprint compared to interpreted languages, such as PHP, NodeJS, and Python
• Easy to learn

There are however also some disadvantages:

• Deployment to different operating systems or CPU architectures requires seperate binaries
• Inferior performance compared to other compiled programming languages without garbage collection, such as C, C++, and Rust
• Garbage collector is non-deterministic and may introduce latency, which can be a problem in low-latency, real-time applications
• No generics — yet

TODO: Describe more why microservices are using NATS and event-based communication via the pub-/sub-pattern. Keywords: Loose coupling, ease of service discovery.

TODO: Elaborate why the Zalando's RESTful API guidelines are used.

TODO: Add note that explains that this repository is not the ideal setup for microservices. Microservices aim for loose coupling and high cohesion. Sharing code and with this also the DRY (don't repeat yourself) paradigm is therefore strongly discouraged. This ensures service stability and prevents building a distributed monolith.

Deployment

This section describes how to deploy the application.

Evaluation

For evaluation and testing the recommended deployment method is docker-compose. Please follow the instructions below to deploy the application locally.

Make sure to have a recent version of Docker CE and Docker Compose installed. You may then open this folder in VS Code or your editor of choice. Upon running the command below the application containers are build and run in daemonized in the background.

docker-compose up -d

Navigate to localtest.me and enjoy.

Tip: You may also omit the -d flag to fun the application in the foreground or run docker-compose logs to view the application logs.

Production

For the deployment into production, we recommend using the container orchestrator Kubernetes. Please follow the steps below to deploy the application.

TODO: Create Helm Chart and describe how to deploy the application to Kubernetes. The Helm Chart repository can be hosted via GitHub pages.

TODO: Describe alternative deployment options for the static web frontend, such as GitHub Pages, GCP Cloud Storage or Azure Static Web Apps.

Operations and security

The scope of this repository is to create a functional prototype. We therefore recommend you to consider the non-exhaustive list below before deploying your application to production:

• Use randomly generated credentials for the databases using the command openssl rand -hex 64 for example
• Expose gateway services via valid TLS certificates using for example Traefik as a reverse proxy and cert-manager with Let's Encrypt for automatic certificate issuing and renewlaexample
• Pin container image versions to make deployments deterministic and thus ensure system stability
• Rebuild and redeploy container images on regular basis to reduce attack surface and ensure that dependencies are up-to-date
• Use a service mesh such as Linkerd to encrypt cluster-internal traffic via mTLS between microservices, especially in multi-tenant clusters
• Set up a monitoring stack using for example Prometheus and Grafana
• Set up a centralized logging stack using for example Grafana Loki
• Deploy services automatically using a CI / CD pipeline provider, such as GitHub Actions
• Define a branching strategy and automate release management using semantic releases
• Configure high availability within Kubernetes and the services

Future improvements

The microservice framework created here could have the following future features:

• Support for message queues via broker.QueueSubscribe() and broker.QueuePublish() or similar
• Logger abstraction via an interface to allow logging to services such as Grafana Loki
• Broker support for Redis and GCP Pub/Sub
• Automatic export of simple prometheus metrics
• Custom method in handlers to manage prometheus metrics
• Aligned handler interface with Knative
• Developer tooling to easily package a microservice
• Benchmark and improve performance and reduce memory usage, possibly via more extensive pointer usage
• WebSocket, MQTT and CoAP gateway
• Auto-configure gateway via Kubernetes API resources Gateway and Ingress

License

This project is licensed under the terms of the MIT License.