- Docker
- kind
- kind is "Kubernetes in Docker," used by the Kubernetes project to help test features and run integration tests. It turns out it's a handy way for anyone to spin up a cluster quickly. Big thank you to @bentheelder for developing it ππΌ ππΌ
- kubectl
- kubectl is how you interact with k8s using the command line. This allows you to become a k8s whisperer :-)
We have crafted a few scripts to make this demo run as quickly as possible on your machine once you've installed the prerequisites.
This script will:
- Create a kind cluster
- Apply the Kubernetes manifests we need for our LLM chatbot demo
- Use port-forwarding to help us access our service in the browser so we can ask our LLM questions
Note: After applying the Kubernetes manifests, our cluster will take roughly 3-4 minutes to download the images, initialize the LLM and start the applications. You can monitor the status of the pods by running
kubectl get pods
./startup.sh
Next, we can install the demo app that uses the LLM.
./demo.sh
If you want to remove or tear down just the demo app, you can run
./teardown.sh
To shut down the demo, run the following command, which will:
- Remove the Kubernetes manifests
- Remove the port-forwarding
- Delete the kind cluster
./shutdown.sh
This demo has been tested on the following operating systems and will work if you have the prerequisites installed.
- macOS
- Linux
- Windows
We'd like to thank Docker for sharing their GenAI Stack demo with us. We've used their demo as a starting point for this demo. You can find their demo here.
We'd also like to thank Weaveworks for their Weaveworks Scope project which we've used to help visualize our Kubernetes cluster. You can find their project here.