queuestat: Empty on 202211
bluecmd opened this issue · comments
Seems to have worked on 202111
but on 202211
it is empty:
(vrf:mgmt)bluecmd@ixp-lab-sw1:~$ queuestat
Port TxQ Counter/pkts Counter/bytes Drop/pkts Drop/bytes
------ ----- -------------- --------------- ----------- ------------
Port TxQ Counter/pkts Counter/bytes Drop/pkts Drop/bytes
------ ----- -------------- --------------- ----------- ------------
Port TxQ Counter/pkts Counter/bytes Drop/pkts Drop/bytes
------ ----- -------------- --------------- ----------- ------------
[..]
Seems the COUNTERS_QUEUE_NAME_MAP
table is empty in COUNTERS_DB
:
127.0.0.1:6379[2]> hgetall COUNTERS_QUEUE_NAME_MAP
(empty array)
In 202111 it is populated.
Seems to be handled by "Flex counter" in SONiC. It seems enabled:
(vrf:mgmt)bluecmd@ixp-lab-sw1:~$ sudo counterpoll show
Type Interval (in ms) Status
-------------------------- ------------------ --------
QUEUE_STAT default (10000) enable
PORT_STAT default (1000) enable
PORT_BUFFER_DROP default (60000) enable
RIF_STAT default (1000) enable
QUEUE_WATERMARK_STAT default (60000) enable
PG_WATERMARK_STAT default (60000) enable
PG_DROP_STAT default (10000) enable
BUFFER_POOL_WATERMARK_STAT default (60000) enable
ACL 10000 enable
I do see the following message in the log:
Feb 28 13:45:07.272066 ixp-lab-sw1 WARNING python3: :- blockable: Key '{COUNTERS_PORT_NAME_MAP}' unavailable in database '{COUNTERS_DB}'
Feb 28 13:45:07.278638 ixp-lab-sw1 WARNING python3: :- blockable: Key '{COUNTERS_PORT_NAME_MAP}' unavailable in database '{COUNTERS_DB}'
Might be related to that we do not have a BUFFER_QUEUE
table in the config?
Maybe related to this feature: https://github.com/sonic-net/SONiC/blob/417a096734f24b6d6ee8e852c74a164410568fe1/doc/SONiC_202205_Release_Notes.md#queue-statistics-based-on-queue-configurations-and-not-max
Although this shouldn't matter, as if the interface is not specified it should default to max number of buffer counters.
Ethernet60 seems to read a sensible default anyway:
Mar 1 15:35:03.837925 ixp-lab-sw1 INFO swss#orchagent: :- initializeQueues: Get 20 queues for port Ethernet60
Mar 1 15:35:03.879777 ixp-lab-sw1 INFO swss#orchagent: :- initializeQueues: Get queues for port Ethernet60
getQueueTypeAndIndex
seems to be called 3 times, but I can find 4 call sites, so maybe the missing one is ours?
Loading the following configuration made queue stats appear for Ethernet68:
{
"BUFFER_POOL": {
"ingress_lossless_pool": {
"size": "32744448",
"type": "ingress",
"mode": "static"
},
"egress_lossy_pool": {
"size": "32744448",
"type": "egress",
"mode": "dynamic"
}
},
"BUFFER_PROFILE": {
"ingress_lossy_profile": {
"pool": "ingress_lossless_pool",
"size": "0",
"static_th": "32744448"
},
"egress_lossy_profile": {
"pool": "egress_lossy_pool",
"size": "0",
"dynamic_th": "3"
}
},
"BUFFER_PG": {
"Ethernet68|0": {
"profile": "ingress_lossy_profile"
}
},
"BUFFER_QUEUE": {
"Ethernet68|0-6": {
"profile": "egress_lossy_profile"
}
}
}
FWIW, this is the queue stats on one of the more active ports on 202111:
Port TxQ Counter/pkts Counter/bytes Drop/pkts Drop/bytes
----------- ----- -------------- --------------- ----------- ------------
Ethernet124 UC0 137424676816 80666063346143 10191 14384228
Ethernet124 UC1 0 0 0 0
Ethernet124 UC2 0 0 0 0
Ethernet124 UC3 0 0 0 0
Ethernet124 UC4 0 0 0 0
Ethernet124 UC5 0 0 0 0
Ethernet124 UC6 0 0 0 0
Ethernet124 UC7 6927623 634638331 0 0
Ethernet124 UC8 0 0 0 0
Ethernet124 UC9 0 0 0 0
Ethernet124 MC10 14848 17854334 0 0
Ethernet124 MC11 0 0 0 0
Ethernet124 MC12 0 0 0 0
Ethernet124 MC13 0 0 0 0
Ethernet124 MC14 0 0 0 0
Ethernet124 MC15 0 0 0 0
Ethernet124 MC16 0 0 0 0
Ethernet124 MC17 1613345 150992262 0 0
Ethernet124 MC18 0 0 0 0
Ethernet124 MC19 0 0 0 0