Ip rate limit issues

[Valdeminas]

Hi.

We have both client and ip rate liming setup on our net 6 app with redis in azure.

This is our configuration:

public static void AddRedisRateLimiting(this IServiceCollection services, IConfiguration configuration)
        {
            var redisOptions = ConfigurationOptions.Parse(configuration["Services_Redis_Secret"]);

            services.AddStackExchangeRedisCache(options =>
            {
                options.Configuration = configuration["Services_Redis_Secret"];
                if (EnvironmentUtility.IsLocalMachine)
                    options.InstanceName = $"{Environment.MachineName}:";
            });

            services.Configure<ClientRateLimitOptions>(configuration.GetSection("UserIdRateLimiting"));
            services.Configure<IpRateLimitOptions>(configuration.GetSection("IpRateLimiting"));
            services.AddSingleton<IRateLimitCounterStore, DistributedCacheRateLimitCounterStore>();
            services.AddDistributedRateLimiting<RedisProcessingStrategy>();

            services.AddSingleton<IConnectionMultiplexer>(provider => ConnectionMultiplexer.Connect(redisOptions));
            services.AddRedisRateLimiting();

            services.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>();
        }

We also call app.UseIpRateLimiting(); and app.UseClientRateLimiting(); further down the line.

Our Ip config looks as follows.

{
  "IpRateLimiting": {
    "EnableEndpointRateLimiting": true,
    "HttpStatusCode": 429,
    "GeneralRules": [
      {
        "Endpoint": "*",
        "Period": "1m",
        "Limit": 80
      }
    ],
    "QuotaExceededResponse": {
      "Content": "{{ \"code\": 429, \"messages\": [\"Whoa! Calm down, cowboy! Quota exceeded. Please try again later.\"], \"correlationId\": \"RateLimit\" }}",
      "ContentType": "application/json",
      "StatusCode": 429
    }
  }
}

We have a similar setup for client, which has a limit of 3req/2s.

We have a 2 tests setup,
One which submits a 100 requests in parallel, and then checks whether any of the responses were rate limited. (for ip rate limit).
Another which submits 10 requests in parallel, and then checks whether any of the responses were rate limited. (for client rate limit).

[Fact]
        [Trait("Category", "IntegrationTest")]
        public async Task ClientRateLimitQuota()
        {
            //Arrange
            var content = new { Message = "rate limiting test" }.CreateStringContent();
            var callsCount = 10;
            var tasks = new List<Task<HttpResponseMessage>>();

            for (int i = 0; i < callsCount; i++)
            {
                tasks.Add(_client.PutAuthenticatedAsync("users/me/description", content, _buyerTokens));
            }

            //Act and assert
            var results = await Task.WhenAll(tasks);

            //Assert
            Assert.Contains(results, x => x.StatusCode == HttpStatusCode.TooManyRequests);
        }

        [Fact]
        [Trait("Category", "IntegrationTest")]
        public async Task IpRateLimitQuota()
        {
            //Arrange
            var content = new { Message = "rate limiting test" }.CreateStringContent();
            var callsCount = 100;

            //Act
            var tasks = new List<Task<HttpResponseMessage>>();
            for (int i = 0; i < callsCount; i++)
            {
                tasks.Add(_client.GetAuthenticatedAsync("appConstants", _buyerTokens));
            }

            var results = await Task.WhenAll(tasks);

            //Assert
            Assert.Contains(results, x => x.StatusCode == HttpStatusCode.TooManyRequests);
        }

Problem is, the client rate limiting seems to work, while the ip rate limiting test fails.

Looking at the logs, it seems that remaining limit counters are being set incorrectly (screenshot attached).
However, it is not clear why the client rate limit test passes.

Could someone consult on this?

Is our configuration correct?
Why is the remaining limit not being set correctly?
Why is client rate limiting not affected?
Is it possible to configure this to work?

Thank you.

[Valdeminas]

Wrote our own middleware ratelimiter with redis