Rate Limiting
About
State of the art rate-limiting in Java. Implemented algorithms:
- Token bucket algorithm
- Fixed window algorithm
- Sliding window log algorithm
Highly customizable and extensible implementation with assumptions about the environment used - it can be easily extended to be used with any key-value storage backend such as:
See Hazelcast or JCache example storage implementation
Features
- Multiple policies per user
- Blazing speed
- Multiple algorithms per user
- Support for distributed environments
- Pluggable storage backend system
- Generic storage key types
Usage
To perform Rate Limiting implement RateLimiter
interface or use existing RateLimiterImpl
. You can implement use your key-value database by implementing StorageBackend
interface or use the existing HazelcastStorage implementation.
Examples
Simple example
StorageBackend<String> storageBackend = new InMemoryStorageBackend<>(); // in memory impl.
EntryStorage entryStorage = new DistributedEntryStorage(storageBackend); // async mode
RateLimiter rateLimiter = RateLimiting.withStorage(entryStorage);
if (rateLimiter.conformsRateLimits("userIdentifier")) {
System.out.println("User has no policies so this will be printed!");
} else {
System.out.println("Too many requests!");
}
Advanced rate limit filter example (javax)
public abstract class RateLimitFilter implements ContainerRequestFilter {
@Override
public void filter(ContainerRequestContext req) throws IOException {
try {
Optional<String> identifier = getIdentifier(req);
if (!identifier.isPresent()) {
return;
}
ConsumptionEntry consumptionEntry = getRateLimiter().conformRateLimitsWithConsumption(identifier.get());
long retryAfter = TimeUnit.NANOSECONDS.toMillis(consumptionEntry.getNanosUntilConsumption());
// Inject headers
response.addHeader(RATE_LIMIT_REMAINING_HEADER,
String.valueOf(consumptionEntry.getRemainingTokens()));
response.addHeader(RETRY_AFTER_HEADER, String.valueOf(retryAfter));
if (!consumptionEntry.doesConform()) {
req.abortWith(createRateLimitResponse(consumptionEntry));
}
} catch (RateLimiterException ex) {
}
}
}
If you need custom serialization combined with your custom storage-backend extend base classes e.g. SimpleRefillPolicy
, AbstractRecord
and AbstractEntry
and implement required serialization methods.
Configuration
Env variables
ratelimit.map.users.limits
: Hazelcast IMap name (defaultratelimit.map.users.limits)
distributedStorageBackendTimeout
: Timeout for rate limiter pass-through mode in ms (default500ms
). You should decrease this in production to avoid long latencies in case of StorageBackend failures.
Scheduling
It turns out rate limiting algorithms are very appropriate for scheduling.
EntryBuilder builder = RateLimiting.schedulerBuilder().withAlgorithm(RateLimitAlgorithm.TOKEN_BUCKET);
RefillPolicy policy = SimpleRefillPolicy.perSecond(2);
RateLimitEntry record = builder.withRefillPolicy(policy).build();
long start = System.currentTimeMillis();
while (record.tryConsume(1)) {
double secondsPassed = (System.currentTimeMillis() - start) / 1000.0;
System.out.println(secondsPassed); // or someVeryExpensiveTask();
}
Output:
0.502
1.004
1.504
2.006
2.508
3.011
...