apache / pulsar

Apache Pulsar - distributed pub-sub messaging system

Home Page:https://pulsar.apache.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Bug] Broker is not blocking the dispatch even after reaching max unack-message limit per subscription

rdhabalia opened this issue · comments

Search before asking

  • I searched in the issues and found nothing similar.

Read release policy

  • I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

After upgrading from Pulsar-2.7 to >= Pulsar-2.10 broker, broker is not considering various broker configuration threshold and one of them is maxUnackedMessagesPerSubscription. We have broker configured with MaxUnackedMessagesPerSubscription=200000 and after sub reaches 200K unack messages, broker blocks the dispatching. It was working fine till broker-2.7 but it stopped working after upgrading to Pulsar-2.10 and higher version.

subscriptions" : {
    "UnackMsgConsumersSubscription" : {
      "msgRateOut" : 41.66667806597534,
      "msgThroughputOut" : 63924.01748854578,
      "bytesOutCounter" : 9813637567183,
      "msgOutCounter" : 6346168888,
      "msgRateRedeliver" : 1825.0005374626583,
      "chuckedMessageRate" : 0,
      "msgBacklog" : 63122123,
      "backlogSize" : 0,
      "msgBacklogNoDelayed" : 63122123,
      "blockedSubscriptionOnUnackedMsgs" : false,
      "msgDelayed" : 0,
      "unackedMessages" : 614260,
      "type" : "Shared",
      "msgRateExpired" : 0.0,
      "totalMsgExpired" : 0,
      "lastExpireTimestamp" : 0,
      "lastConsumedFlowTimestamp" : 1712610077925,
      "lastConsumedTimestamp" : 1712610077938,
      "lastAckedTimestamp" : 1712610088862,
      "lastMarkDeleteAdvancedTimestamp" : 1712609791878,
      "consumers" : [ {
        "msgRateOut" : 41.66667806597534,
        "msgThroughputOut" : 63924.01748854578,
        "bytesOutCounter" : 5106838658,
        "msgOutCounter" : 3324500,
        "msgRateRedeliver" : 1825.0005374626583,
        "chuckedMessageRate" : 0.0,
        "consumerName" : "0f455",
        "availablePermits" : 0,
        "unackedMessages" : 614260,
        "avgMessagesPerEntry" : 1,
        "blockedConsumerOnUnackedMsgs" : false,
        "lastAckedTimestamp" : 1712610088862,
        "lastConsumedTimestamp" : 1712610077938,
        "metadata" : { },
        "connectedSince" : "2024-04-08T20:52:27.525995Z",

Minimal reproduce step

  1. Create namespace and topics with version <= Pulsar-2.7
  2. Upgrade broker 2.10 and configure maxUnackedMessagesPerSubscription=200000
  3. Create sub with unack messages more than 200K and broker will still not block the dispatching

What did you expect to see?

Broker should follow max threshold for various resources

What did you see instead?

Broker is not following max threshold for various resources

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!