animir / node-rate-limiter-flexible

Atomic counters and rate limiting tools. Limit resource access at any scale.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

npm version npm node version deno version

Logo

node-rate-limiter-flexible

rate-limiter-flexible counts and limits number of actions by key and protects from DDoS and brute force attacks at any scale.

It works with Redis, Prisma, DynamoDB, process Memory, Cluster or PM2, Memcached, MongoDB, MySQL, _PostgreSQL.

Memory limiter also works in browser.

Atomic increments. All operations in memory or distributed environment use atomic increments against race conditions.

Fast. Average request takes 0.7ms in Cluster and 2.5ms in Distributed application. See benchmarks.

Flexible. Combine limiters, block key for some duration, delay actions, manage failover with insurance options, configure smart key blocking in memory and many others.

Ready for growth. It provides unified API for all limiters. Whenever your application grows, it is ready. Prepare your limiters in minutes.

Friendly. No matter which node package you prefer: redis or ioredis, sequelize/typeorm or knex, memcached, native driver or mongoose. It works with all of them.

In memory blocks. Avoid extra requests to store with inMemoryBlockOnConsumed.

Allow traffic bursts with BurstyRateLimiter.

Deno compatible See this example

It uses fixed window as it is much faster than rolling window. See comparative benchmarks with other libraries here

Installation

npm i --save rate-limiter-flexible

yarn add rate-limiter-flexible

Import

// CommonJS
const { RateLimiterMemory } = require("rate-limiter-flexible");

// or

// ECMAScript 
import { RateLimiterMemory } from "rate-limiter-flexible";
// or
import RateLimiterMemory from "rate-limiter-flexible/lib/RateLimiterMemory.js";

Basic Example

Points can be consumed by IP address, user ID, authorisation token, API route or any other string.

const opts = {
  points: 6, // 6 points
  duration: 1, // Per second
};

const rateLimiter = new RateLimiterMemory(opts);

rateLimiter.consume(remoteAddress, 2) // consume 2 points
    .then((rateLimiterRes) => {
      // 2 points consumed
    })
    .catch((rateLimiterRes) => {
      // Not enough points to consume
    });

RateLimiterRes object

Both Promise resolve and reject return object of RateLimiterRes class if there is no any error. Object attributes:

RateLimiterRes = {
    msBeforeNext: 250, // Number of milliseconds before next action can be done
    remainingPoints: 0, // Number of remaining points in current duration 
    consumedPoints: 5, // Number of consumed points in current duration 
    isFirstInDuration: false, // action is first in current duration 
}

You may want to set next HTTP headers to response:

const headers = {
  "Retry-After": rateLimiterRes.msBeforeNext / 1000,
  "X-RateLimit-Limit": opts.points,
  "X-RateLimit-Remaining": rateLimiterRes.remainingPoints,
  "X-RateLimit-Reset": new Date(Date.now() + rateLimiterRes.msBeforeNext)
}

Advantages:

Full documentation is on Wiki

Middlewares, plugins and other packages

Some copy/paste examples on Wiki:

Migration from other packages

  • express-brute Bonus: race conditions fixed, prod deps removed
  • limiter Bonus: multi-server support, respects queue order, native promises

Docs and Examples

Changelog

See releases for detailed changelog.

Basic Options

  • points

    Default: 4

    Maximum number of points can be consumed over duration

  • duration

    Default: 1

    Number of seconds before consumed points are reset.

    Never reset points, if duration is set to 0.

  • storeClient

    Required for store limiters

    Have to be redis, ioredis, memcached, mongodb, pg, mysql2, mysql or any other related pool or connection.

Other options on Wiki:

Smooth out traffic picks:

Specific:

API

Read detailed description on Wiki.

Benchmark

Average latency during test pure NodeJS endpoint in cluster of 4 workers with everything set up on one server.

1000 concurrent clients with maximum 2000 requests per sec during 30 seconds.

1. Memory     0.34 ms
2. Cluster    0.69 ms
3. Redis      2.45 ms
4. Memcached  3.89 ms
5. Mongo      4.75 ms

500 concurrent clients with maximum 1000 req per sec during 30 seconds

6. PostgreSQL 7.48 ms (with connection pool max 100)
7. MySQL     14.59 ms (with connection pool 100)

Note, you can speed up limiters with inMemoryBlockOnConsumed option.

Contribution

Appreciated, feel free!

Make sure you've launched npm run eslint before creating PR, all errors have to be fixed.

You can try to run npm run eslint-fix to fix some issues.

Any new limiter with storage have to be extended from RateLimiterStoreAbstract. It has to implement 4 methods:

  • _getRateLimiterRes parses raw data from store to RateLimiterRes object.

  • _upsert may be atomic or non-atomic upsert (increment). It inserts or updates value by key and returns raw data. If it doesn't make atomic upsert (increment), the class should be suffixed with NonAtomic, e.g. RateLimiterRedisNonAtomic.

    It must support forceExpire mode to overwrite key expiration time.

  • _get returns raw data by key or null if there is no key.

  • _delete deletes all key related data and returns true on deleted, false if key is not found.

All other methods depends on store. See RateLimiterRedis or RateLimiterPostgres for example.

Note: all changes should be covered by tests.

About

Atomic counters and rate limiting tools. Limit resource access at any scale.

License:ISC License


Languages

Language:JavaScript 100.0%