How to handle if can't connect to Redis cluster ?
x34599 opened this issue · comments
Hey @x34599. I am also interesting in the disabling rate limiting in case of redis unavailability. Do you have any updates about this issue?
Hey @x34599. I am also interesting in the disabling rate limiting in case of redis unavailability. Do you have any updates about this issue?
@DmitriyVorobey Nope, I still not get any update. But I found it has 2 case about this
-
Can't connect to redis since API started.
I tried many way to handle this and I found this work (for me)
in Startup.cs
inConfigureServices
add this
inConfigure
use middleware like this
-
Can't connect to redis when your API is running.
I just addawait _next(context);
inRedisException
of previous middleware and it work normally.
TO BE HONEST I don't know is this a proper way to handle, If you found any problem or have any suggestion please let me know.
Thanks @x34599 for sharing your solution. That works fine, but there is an edge case. It doesn`t handle situation when connection is lost when app is running. As alternative solution I decided to wrap IpRateLimitMiddleware middleware via custom middleware and check is connection available on each request. Custom middleware allow to:
- Disable rate limiting if connection is lost.
- Enable rate limiting if connection is restored.
I am also not 100% sure that it`s the best solution. I will be happy to hear a feedback.
public class IpOptionalRedisRateLimitMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger<IpRateLimitMiddleware> _logger;
private readonly IConnectionMultiplexer _connectionMultiplexer;
private readonly IpRateLimitMiddleware _rateLimitMiddleware;
public IpOptionalRedisRateLimitMiddleware(RequestDelegate next,
IProcessingStrategy processingStrategy,
IOptions<IpRateLimitOptions> options,
IRateLimitCounterStore counterStore,
IIpPolicyStore policyStore,
IRateLimitConfiguration config,
ILogger<IpRateLimitMiddleware> logger,
IConnectionMultiplexer connectionMultiplexer)
{
_next = next;
_logger = logger;
_connectionMultiplexer = connectionMultiplexer;
_rateLimitMiddleware = new IpRateLimitMiddleware(next, processingStrategy,
options, counterStore,
policyStore, config, logger);
}
public async Task Invoke(HttpContext context)
{
if(!_connectionMultiplexer.IsConnected)
{
_logger.LogError("Could not connect to redis. Rate limiting disabled.");
await _next.Invoke(context);
return;
}
await _rateLimitMiddleware.Invoke(context);
}
}
Thanks @x34599 for sharing your solution. That works fine, but there is an edge case. It doesn`t handle situation when connection is lost when app is running. As alternative solution I decided to wrap IpRateLimitMiddleware middleware via custom middleware and check is connection available on each request. Custom middleware allow to:
- Disable rate limiting if connection is lost.
- Enable rate limiting if connection is restored.
I am also not 100% sure that it`s the best solution. I will be happy to hear a feedback.
public class IpOptionalRedisRateLimitMiddleware { private readonly RequestDelegate _next; private readonly ILogger<IpRateLimitMiddleware> _logger; private readonly IConnectionMultiplexer _connectionMultiplexer; private readonly IpRateLimitMiddleware _rateLimitMiddleware; public IpOptionalRedisRateLimitMiddleware(RequestDelegate next, IProcessingStrategy processingStrategy, IOptions<IpRateLimitOptions> options, IRateLimitCounterStore counterStore, IIpPolicyStore policyStore, IRateLimitConfiguration config, ILogger<IpRateLimitMiddleware> logger, IConnectionMultiplexer connectionMultiplexer) { _next = next; _logger = logger; _connectionMultiplexer = connectionMultiplexer; _rateLimitMiddleware = new IpRateLimitMiddleware(next, processingStrategy, options, counterStore, policyStore, config, logger); } public async Task Invoke(HttpContext context) { if(!_connectionMultiplexer.IsConnected) { _logger.LogError("Could not connect to redis. Rate limiting disabled."); await _next.Invoke(context); return; } await _rateLimitMiddleware.Invoke(context); } }
@DmitriyVorobey
Sorry for my unclear answer previously, actually in 2nd case I added all of a RedisException (RedisConnectionException, RedisServerException, RedisTimeoutException, RedisCommandException, RedisException) in previous middleware and it will skip rate limit validation when can't connect to redis same as your solution
catch (RedisConnectionException ex)
{
_logger.Error("RedisConnectionException ");
await _next(context);
}
catch (RedisServerException ex)
{
_logger.Error("RedisServerException ");
await _next(context);
}
catch (RedisTimeoutException ex)
{
_logger.Error("RedisTimeoutException ");
await _next(context);
}
catch (RedisCommandException ex)
{
_logger.Error("RedisCommandException");
await _next(context);
}
catch (RedisException ex)
{
_logger.Error("RedisException");
await _next(context);
}
as I said earlier I still not sure it's proper solution and still concern about it (will it create tech. debt or it will make your api slower).