ReadySet is a SQL caching engine designed to help developers enhance the performance and scalability of their existing database-backed applications.
What is ReadySet?
ReadySet is a lightweight SQL caching engine that precomputes frequently-accessed query results and automatically keeps these results up-to-date over time as the underlying data in your database changes. ReadySet is wire-compatible with MySQL and Postgres and can be adopted without code changes.
ReadySet acts as both a SQL cache and a proxy. When you first connect ReadySet to your application, it defaults to proxying all of your queries to your backing database, so it doesn't change the behavior of your application.
From there, you can monitor the performance of your queries using the ReadySet dashboard, and cache a subset of them in ReadySet. ReadySet automatically keeps cached results up-to-date over time as the underlying data in your database changes, so there's no need to write cache maintenance logic.
Getting Started
Integrating with ReadySet is straight-forward:
- Deploy ReadySet and connect it to your MySQL or Postgres database.
- Change your application's connection string to point at ReadySet.
- Profile queries in the ReadySet Dashboard.
- Cache queries using ReadySet's custom SQL commands.
To run through this process on your local machine, see the Quickstart.
To run through this process in a cloud environment, see Deploy with Kubernetes or Deploy with ReadySet Cloud.
FAQs
Q: How does ReadySet work under the hood?
A: The heart of ReadySet is a query engine based on partially-stateful, streaming data flow. To learn more about how it works, see our docs.
Q: How does ReadySet keep cached state up to date?
A: ReadySet receives updates about data changes from your backing database via binlog replication and uses those updates to automatically update its internal state.
Q: Do I have to send all of my database traffic to ReadySet?
A: You can if you want to, but it’s not required. You can manually route a subset of your traffic to ReadySet (as you would with a traditional database read replica), or you can send all of it to ReadySet. It's important to note that not all of the queries sent to ReadySet need to be cached– you have fine-grained control over what is cached vs. what is proxied.
Documentation
For more information, check out our full docs site here.
Join the Community
For questions or support, join us on the ReadySet Community Discord, post questions on our Github forum, or schedule an office hours chat with our team.
Everyone is welcome!
Development
Prerequisites
Install Dependencies
Prior to running ReadySet, you may need to install the following dependencies: clang libclang-dev libssl-dev liblz4-dev build-essential
macOS:
brew install lz4
brew install openssl@1.1
Ubuntu:
sudo apt update && sudo apt install -y build-essential libssl-dev pkg-config llvm clang liblz4-dev cmake
Install Rust
ReadySet is written entirely in Rust. If you don’t already have Rust installed, you can install it via rustup (select the version of ‘nightly’ specified in the rust-toolchain file):
curl https://sh.rustup.rs -sSf | sh
Runtime Dependencies: Upstream Database and Consul
Local Development
To streamline local development, all runtime dependencies (Consul, MySQL, Postgres) can be run with:
cp docker-compose.override.yml.example docker-compose.override.yml
docker-compose up -d
Full Deployment
ReadySet runs alongside a backing MySQL or Postgres database and uses Consul for leader election. You’ll need to pass in your database’s connection string when running the ReadySet server and adapter, and have a Consul instance running.
Note: Consul persists data per deployment. Using the same deployment name will carry over any previously installed queries into a new deployment.
Run ReadySet
ReadySet Server
First, compile and run ReadySet server.
cargo run --bin readyset-server --release -- --upstream-db-url <upstream-url> --deployment <deployment name>
If using the databases supplied by the docker-compose environment in this repository, replace with the URL of the database corresponding to your database engine:
- MySQL:
mysql://root:readyset@127.1/readyset - PostgreSQL:
postgresql://root:readyset@127.1/readyset
If running with an existing external database, replace with the connection string for that database.
ReadySet MySQL or Postgres Adapter
Then, run the adapter binary corresponding to your upstream database (MySQL or Postgres) The adapter will communicate with servers that have the same deployment name.
MySQL
cargo run --bin readyset-mysql --release -- --upstream-db-url mysql://root:readyset@127.1/readyset --allow-unauthenticated-connections
--address 0.0.0.0:3307 --deployment <deployment name> --prometheus-metrics --query-log --query-log-ad-hoc
Postgres
cargo run --bin readyset-psql --release -- --upstream-db-url postgresql://postgres:readyset@127.1/readyset --allow-unauthenticated-connections
--address 0.0.0.0:5433 --deployment <deployment name> --prometheus-metrics --query-log --query-log-ad-hoc
The adapter listens for connections at the address specified in the address flag.
The query-log and query-log-ad-hoc flags ensure that queries are sent to the Prometheus client running in the adapter.
The prometheus-metrics flag exposes an HTTP endpoint in the adapter to allow querying of metrics. This can be reached with an HTTP GET request to :6034/prometheus (e.g., curl -X GET 127.0.0.1:6034/prometheus).
Testing
To run tests for the project, run the following command:
cargo test --skip integration_serial
Testing Notes
Certain tests cannot be run in parallel with others, and these tests are typically in files postfixed with _serial. Running the entire set of tests for a package, i.e. cargo test -p readyset-server may fail if serial tests are included.
Running tests may require increasing file descriptor limits. You can do so by running ulimit -Sn 65535.
Performance
When running ReadySet in a performance-critical setting, make sure you compile with the --release flag.
Note: This repository contains a snapshot of the current state of the ReadySet internal repository. Our team is currently working on getting our live codebase, including all development history, ready to be hosted on GitHub, at which point we'll force push to this repository.
License
ReadySet is licensed under the BSL 1.1 license, converting to the open-source Apache 2.0 license after 4 years. It is free to use on any number of nodes. The ReadySet team is hard at work getting the codebase ready to be hosted on Github.
ReadySet is also available as a paid cloud service that can be deployed to your accounts or hosted entirely by ReadySet. You can get early access to our cloud product by signing up here.