questdb / questdb-packer-ami

Scripts and resources that can be used to build your own AMI with QuestDB.

QuestDB AMI Build Specification

This repository contains scripts and resources that can be used to build your own AMI with QuestDB. The AMI is based on Amazon Linux 2 and generated using Packer from HashiCorp.

Prerequisites

In order to use this template, you will need:

• Packer
• AWS credentials

Repository structure

└── src
    ├── assets
    │   ├── bashrc          # Used to set 'JAVA_HOME'
    │   ├── cloudwatch.json # Configuration which enables logs forwarding to AWS CloudWatch
    │   ├── logrotate.conf  # Configuration file for logrotate
    │   ├── rsyslog.conf    # Configuration file for rsyslog
    │   └── systemd.service # Service file to run QuestDB via systemd
    ├── config
    │   └── server.conf     # Configuration file for QuestDB
    ├── scripts
    │   └── per-boot.sh     # Script that runs every time the instance is started (CloudWatch)
    ├── build.bash          # Script that automates the installation of all the components on the AMI
    └── template.json       # Template file for Packer

Build the AMI

1. Clone this repository:

git clone https://github.com/questdb/questdb-packer-ami.git

2. Navigate to src and run packer:

cd questdb-packer-ami/src
packer build template.json

That's it, you can now create a new EC2 instance using the AMI you just built.

Usage

Configuration

The AMI built using this template is very generic. You might want to change the database configuration file:

src/config/server.conf

For all the properties and values that you can set, please check our documentation.

Logs and AWS CloudWatch

This AMI uses logrotate to automatically trim and archive logs generated by QuestDB. The AWS CloudWatch agent is also preinstalled and already configured. If you want the logs to be available on your CloudWatch dashboard, you need to:

1. Make sure you run your EC2 with an instance profile that has the CloudWatchAgentServerPolicy IAM policy
2. Create the necessary CloudWatch resources: /questdb-<instance-id> (with leading slash) log group and questdb-systemd-service.log log stream (example of instance id: i-0c1386329d00506a2)

Resources

Complete references are available in the Documentation.

Get started:

• Docker
• Binaries
• Homebrew

Develop:

• Connect
• Insert data
• Query data
• Authenticate

Concepts:

• SQL extensions
• Storage model
• Partitions
• Designated timestamp

Support / Contact

Slack channel

Roadmap

Our roadmap is here

Contribution

Feel free to contribute to the project by forking the repository and submitting pull requests. Please make sure you have read our contributing guide.