Transparently send requests from one URL to another with output comparison.
Test for API compatibility, of any size change, to HTTP endpoints without expensive manual testing or disrupting production traffic. This is intended to complement automated testing suites by bringing that extra few feet of confidence in changes.
overview.mp4
- Little to no overhead π€
- Shadowing occurs after the original request's response is sent back to the client--keeping your latency sensitive services happy
- Flexible configuration π¨
- Combine Workers'
routes
configuration and JavaScript
- Combine Workers'
- At-rest data encryption π
- Replay requests π
- Automatic grouping π₯
- Tagging π·οΈ
- Export π¦
- Themed interface βοΈπ
- Sharable links (group work in mind)
Compare JSON responses without inconsequential diffs.
- Objects: Properties can be moved but their value cannot change
- Arrays: Entries cannot move or change value
- Moves are tracked separately from deletions/additions
Screenshots πΈ
diff.mp4
Visualize divergence trends with aggregated data through the UI or API.
Quickly see what class of issue is happening most.
Groups are created for each unique set of divergent response keys. So, given:
- Response of shadow request A has 2 divergent keys
name
andprice
- Response of shadow request B has 2 divergent keys
name
andprice
- Response of shadow request C has 1 divergent key
name
- Response of shadow request D has no divergent keys
We would have 2 groups:
- π₯ Request A and B --
name
andprice
- π₯ Request C --
name
Request D is not given a group or rendered on the page as it isn't divergent. It will be included in the aggregation graph under "Total"s though.
Quickly export saved responses for use fixtures elsewhere.
Apply tags you can filter by using the UI or API. Computed with JavaScript, you have the flexibility to create effective tags for your use-case.
We try to make anything intractable translate to the URL so you can easily share what you're seeing with coworkers.
Comfortably process requests knowing exactly what code is running with at-rest encryption* of sensitive content. Especially useful in regulated environments.
- Control
- Request headers π
- Response body π
- Response headers are not saved
- Shadows
- Request URL π«
- Though encrypted in-transit by TLS, we consider URLs as low sensitivity content and save it in plain-text. Do not put sensitive content in URLs!
- Request method π«
- Request headers π
- Response body ππ«
- We save which paths diverge in plain-text for performant lists and grouping. Everything else is encrypted.
- For example, if the control response and shadow response's
.name
properties diverge,['name']
would be saved in plain-text while the full value is encrypted.
- For example, if the control response and shadow response's
- We save which paths diverge in plain-text for performant lists and grouping. Everything else is encrypted.
- Response headers π
- Response status code π«
- Request URL π«
- Tags π«
See schema/table for a rough idea on data structure
* Using a 256 bit AES-GCM key derived, from a secret of your choice, using PBKDF2. See source code for implementation.
Systems can be complex and indeterminate. Replays allow you to resend requests ad-hoc to help track down flaky mismatches.
Replays trigger a request to the same URL and headers that triggered the original shadow. This triggers a shadow as usual but the result will be saved to the shadow you triggered the replay from instead of creating a new one.
Supporting both people who like to actually read whats on their monitor with bright lights around and those who won't accept anything but a dark mode (or to avoid late night flash bang outs)
Page theme follows system/browser theme
Note
You'll need to use Cloudflare as a reverse proxy1 to run this!
You are responsible for deploying and operating this tool. I'll do what I can to answer questions and provide guides though. π
There are 3 runtime components:
shadower
: Forwards original (aka control) requests, sends shadow request, compares responses, and saves output to databaseapi
: Pull records from database- Note: I had originally only separated this as Pages' Workers struggled with Node.js compatibility on (for
pg
module) which may no longer be the case. At this point, I like the boundary.
- Note: I had originally only separated this as Pages' Workers struggled with Node.js compatibility on (for
web
: Front-end toapi
rendering diffs (skip for bring-your-own-interface)
What to bring:
- Postgres server
- Any reasonably recent version should do
jsonb
+ its operators are the only "hasn't been in Postgres for a few decades" features in use
- Sizing is relative to expected load
- Anecdotally: We've been running AWS' Aurora Serverless with 4 APU at 4/rps (20/rps burst) without breaking 25% database load.
- Any reasonably recent version should do
- Cloudflare account
Steps:
- Git clone or download project
- Setup Cloudflare Access for the domain you'll host the web interface on
- This will likely be
project-name.pages.dev
whereproject-name
is indeploy
of these scripts
- This will likely be
- Create database, table, and user
npm ci
: Install setup script dependenciesnode setup.mjs
: Run setup script- Alternatively, do what setup.mjs is doing by hand
- Adjust
getShadowingConfigForUrl
in shadower- See option documentation under
ShadowingConfig
type
- See option documentation under
- Adjust
routes
in shadower wrangler.toml - Deploy shadower (
npm run deploy
inshadower
)
- Create database:
CREATE DATABASE request_shadowing
- Verify
gen_random_uuid
is available:SELECT gen_random_uuid();
- Enable
uuid-ossp
if not:CREATE EXTENSION uuid-ossp
- Enable
- Create tables and indices
- Run tables.sql
- Open up an issue with your use-case and index if you end up adding your own/replacing the out-of-box ones! π
- Create user
- Least amount of privileges possible. It doesn't do anything special.
- Setup permissions for user
GRANT SELECT, INSERT, UPDATE, REFERENCES ON requests TO user;
- Good to go!
Footnotes
-
Verify there is an "orange cloud" on the dashboard for the domain you intend to use. See docs/orange-cloud.png. β©