gRPC does not update the BGP data
littlespace opened this issue · comments
Hi I am testing the gRPC with multiple router (QFX10K) with routing table more than 500K
and I am using the "/network-instances/network-instance/protocols/protocol/bgp/" sensor and the sample_frequency = "5000ms" to get the BGP data back from the router. the issue i can not see the data in the DB every 5s. for example right now I am trying to get the "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/afi-safis/afi-safi/state/prefixes/accepted" and it is 1:15pm and the data in the database that I have is:
10:54am - 707K
11:43am - 706K
11:45am - 706K
and between 11:45am - 1:15pm I do not see any data in the DB.
Please let me know how can I fix this issue.
Thanks
Hi,
Could you please share the telegraf config you are using ?
I'm running a quick (2 vMX ~65k routes ) and so far it's working fine
select "/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/afi-safis/afi-safi/state/prefixes/accepted" from "/network-instances/network-instance/protocols/protocol/bgp/" group by device order by desc limit 10
name: /network-instances/network-instance/protocols/protocol/bgp/
tags: device=10.102.183.182
time /network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/afi-safis/afi-safi/state/prefixes/accepted
018-09-24T08:15:10.642309537Z 65536
018-09-24T08:15:05.642649618Z 65536
018-09-24T08:15:00.664520859Z 65536
018-09-24T08:14:55.651129288Z 65536
018-09-24T08:14:50.640652807Z 65536
018-09-24T08:14:45.650554505Z 65536
018-09-24T08:14:40.636820506Z 65536
018-09-24T08:14:35.649300171Z 65536
018-09-24T08:14:30.646534748Z 65536
018-09-24T08:14:25.653037526Z 65536
ame: /network-instances/network-instance/protocols/protocol/bgp/
ags: device=10.102.183.150
ime /network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor/ fi-safis/afi-safi/state/prefixes/accepted
018-09-24T08:15:10.959130531Z 0
018-09-24T08:15:05.924139874Z 0
018-09-24T08:15:00.909217459Z 0
018-09-24T08:14:55.936040729Z 0
018-09-24T08:14:50.906922955Z 0
018-09-24T08:14:45.916258119Z 0
018-09-24T08:14:40.924186838Z 0
018-09-24T08:14:35.942446021Z 0
018-09-24T08:14:30.907905893Z 0
Thanks, i will do that. Also I would like to know If there was a reason that you didn’t use only one tool for collecting everything? Like using only fluentd or just telegraf for colllecting jti, grpc,...
Thanks
Actually, we are working on having a "full matrix" of plugins (fluentd/telegraf/logstash) so that you can decide which one to use.
I am trying to do UDP JTI with Telegraf. would you mind share your finding?
Hi ,
clone my personal repo if you want to take a look. Bear in mind that is still in development phase.
1) Remove old images from openNTI
2) git clone https://github.com/psagrera/open-nti.git -b jvision-telegraf
3) make build
4) make start
Regards
Thanks @psagrera how can I get the juniperUDP input plugin working? are you using json and protoc for this?
Actually, juniperUDP is a parser. We are using input_socket listener plugin to receive UDP traffic. (json ,jsonpb and proto is used). But as I said it is not the definitive version.
Regards
thanks I got it working. one more question right now the data for the juniperUDP plugin is all string even for interface data. how can we change that so we can graph the juniperUDP plug data?
Forgive me for dropping in on your thread....you all seem knowledgeable about this telem stuff, so perhaps you can guide me.
I'm just recently learning about jti, analytics and telemetry. I recently got my MX960's sending native telemetry udp/gpb to open-nti docker (which as you all may well know is a packaged install containing all those things like fluentd, influxdb, chronograph, grafana)... it's all quite interesting to me and working nicely.
i see that there are about ~15 different sensor resources to collect analytics from....
agould@lab-960# set services analytics sensor test resource /junos/s?
Possible completions:
System resource identifier string
/junos/services/cube/usage/
/junos/services/ip-tunnel/usage/ PFE sensor for IP Tunnel statistics
/junos/services/label-switched-path/usage/ PFE sensor for LSP statistics
/junos/services/mgmt-cube/usage/
/junos/services/segment-routing/interface/egress/usage/ PFE sensor for SR per-IF egress stats
/junos/services/segment-routing/interface/ingress/usage/ PFE sensor for SR per-IF ingress stats
/junos/services/segment-routing/sid/usage/ PFE sensor for SR per-SID ingress stats
/junos/system/cmerror/configuration/ Error monitoring configuration
/junos/system/cmerror/counters/ Error monitoring counters
/junos/system/linecard/cpu/memory/ PFE sensor for CPU (ukernel) memory
/junos/system/linecard/fabric/ PFE sensor for fabric traffic
/junos/system/linecard/firewall/ PFE sensor for firewall filter stats/memory
/junos/system/linecard/interface/ PFE sensor for physical interface traffic
/junos/system/linecard/interface/logical/usage/ PFE sensor for logical interface traffic
/junos/system/linecard/intf-exp/ PFE sensor for physical interface express statistics
/junos/system/linecard/npu/memory/ PFE sensor for NPU memory
/junos/system/linecard/npu/utilization/ PFE sensor for NPU utilization
/junos/system/linecard/optics/ PFE sensor for optics diagnostics
/junos/system/linecard/packet/usage/ PFE sensor for packet statistics
/junos/system/linecard/qmon/ Hardware based queue-monitoring
/junos/system/linecard/services/inline-jflow/ PFE sensor for inline-jflow service stats/memory
question please...
1 - are there any more than this ? or is this a limitation of Junos: 17.4R1-S2.2 ? will more sensors come in the future ? i see you all referring to what appear to be sensor resources like
2 - i understand there is a different way of doing telemetry, correct ? is this known as the non-native form ... gRPC/openconfig or something like that ? /network-instances/network-instance/protocols/protocol/bgp/ ...but i don't see those in my MX960.
3 - do some juniper platforms like ACX5048 not support native telem, but may infact support grpc/openconfig style ?
Thanks y'all
- Aaron
Hi Aaron,
1 - More sensors will come in the future (newer releases ) i.e ("/junos/services/segment-routing/traffic-engineering/ingress/usage/", etc ...)
2 - You have UDP based native sensors and OpenConfig + GRPC sensors. Former are GPB encoded and configured via junos cli , latter are GBP / key, value encoded and you need to configure your collector (open-NTI via telegraf config) to subscribe to those paths. Information about sensors can be found at :
https://www.juniper.net/documentation/en_US/junos/topics/reference/general/junos-telemetry-interface-grpc-sensors.html
3 - Not supported, according to the official documentation
Regards
Pablo
Hi @psagrera I am using the new parser.go and it was working for 2-3 weeks but starting today i am getting this error:
panic: interface conversion: interface {} is map[string]interface {}, not []interface {}
goroutine 330 [running]:
github.com/influxdata/telegraf/plugins/parsers/juniperUDP.(*JuniperUDPParser).Parse(0xc4202800d8, 0xc420968000, 0x192, 0x10000, 0x192, 0x1ab4bc0, 0xc420856750, 0x0, 0x0)
/home/ubuntu/go/src/github.com/influxdata/telegraf/plugins/parsers/juniperUDP/parser.go:275 +0x1e8e
github.com/influxdata/telegraf/plugins/inputs/socket_listener.(*packetSocketListener).listen(0xc4204326a0)
/home/ubuntu/go/src/github.com/influxdata/telegraf/plugins/inputs/socket_listener/socket_listener.go:152 +0x2cb
created by github.com/influxdata/telegraf/plugins/inputs/socket_listener.(*SocketListener).Start
/home/ubuntu/go/src/github.com/influxdata/telegraf/plugins/inputs/socket_listener/socket_listener.go:303 +0x21c
I have not change anything on the server or router side. also gRPC is not working in this case even when I disable the new JniperUDP parser. any thought?