Error "error walking target x.x.x.x: request timeout (after 3 retries)"

Question

Error "error walking target x.x.x.x: request timeout (after 3 retries)"

wellsgz opened this issue 3 years ago · comments

While trying to scrape Mikrotik target, in walking subtree" oid=1.3.6.1.4.1.14988, snmp_exporter returns the following error message:

Error scraping target" err="error walking target x.x.x.x: request timeout (after 3 retries)

I also try snmpwalk (snmpwalk -v 2c -c public x.x.x.x 1.3.6.1.4.1.14988) the target with subtree 1.3.6.1.4.1.14988 and encounter "No more variables" error

[snipped]
SNMPv2-SMI::enterprises.14988.1.1.17.1.1.4.1 = STRING: "RouterOS v6.48.3 May/25/2021 06:09:45"
SNMPv2-SMI::enterprises.14988.1.1.17.1.1.5.1 = INTEGER: 1
SNMPv2-SMI::enterprises.14988.1.1.17.1.1.6.1 = INTEGER: 1
SNMPv2-SMI::enterprises.14988.1.1.20.1.1.0 = Gauge32: 0
SNMPv2-SMI::enterprises.14988.1.1.20.2.1 = No more variables left in this MIB View (It is past the end of the MIB tree)

I did some search but can't figure out what causes the issue. Would you mind shedding me some light?

IgorKha · Answer 1 · Tue Jul 20 2021 00:18:58 GMT+0800 (China Standard Time)

SNMPv2-SMI::enterprises.14988.1.1.20.2.1 = No more variables left in this MIB View (It is past the end of the MIB tree)

1.3.6.1.4.1.14988.1.1.20.2.1.* Is responsible for IKE security association. Should be displayed when ipsec is running, but I haven't tested that

Error scraping target" err="error walking target x.x.x.x: request timeout (after 3 retries)

never seen such a issue. I use SNMP v3 trap in my mikrotik and snmp_exporter 0.20.0
Try running snmp_exporter from my repository, he 100% work and open in you browser http://host.ip.address:9116/snmp?module=mikrotik&target=mikrotik.ip.address

IgorKha · Answer 2 · Tue Jul 20 2021 00:27:04 GMT+0800 (China Standard Time)

yes, I tested 1.3.6.1.4.1.14988.1.1.20.2.1.* works if ipsec connect is active

wellsgz · Answer 3 · Sun Jul 25 2021 20:57:23 GMT+0800 (China Standard Time)

I guess the problem may due to lack of IPsec conn in my setup. I have masked 1.3.6.1.4.1.14988 in snmp.yml and the rest works well.

IgorKha · Answer 4 · Mon Jul 26 2021 05:17:53 GMT+0800 (China Standard Time)

this is not error
do not need to masked, 1.3.6.1.4.1.14988 does not interfere with the work even if there is no data in this OID

auterpring · Answer 5 · Mon Jan 31 2022 19:08:54 GMT+0800 (China Standard Time)

I have a same issue
http://localhost:9116/snmp?target=192.168.1.254&module=mikrotik

output: `An error has occurred while serving metrics:

error collecting metric Desc{fqName: "snmp_error", help: "Error scraping target", constLabels: {}, variableLabels: []}: error walking target 192.168.1.254: request timeout (after 3 retries)`

prometheus config:
`scrape_configs:

job_name: 'prometheus'
scrape_interval: 5s

static_configs:
- targets: ['localhost:9090']
job_name: 'Mikrotik'
static_configs:
- targets:
  - 192.168.1.254 # SNMP device.
    metrics_path: /snmp
    params:
    module: [mikrotik]
    relabel_configs:
- source_labels: [address]
  target_label: __param_target
- source_labels: [__param_target]
  target_label: instance
- target_label: address
  replacement: localhost:9116
  `
  Targets:

IgorKha · Answer 6 · Mon Jan 31 2022 20:41:04 GMT+0800 (China Standard Time)

@auterpring
maybe you have 7.x version of rOS installed ?
In version 7, delays in the response to SNMP on weak models were noticed

auterpring · Answer 7 · Mon Jan 31 2022 20:44:01 GMT+0800 (China Standard Time)

@auterpring maybe you have 7.x version of rOS installed ? In version 7, delays in the response to SNMP on weak models were noticed

No, I have 6.48.3. RouterOS version.
BTW, when I use MIB Browser, I can get all information and OIDs

auterpring · Answer 8 · Mon Jan 31 2022 20:48:24 GMT+0800 (China Standard Time)

@auterpring maybe you have 7.x version of rOS installed ? In version 7, delays in the response to SNMP on weak models were noticed

in Mikrotik log I can see, that exporter is working

g

IgorKha · Answer 9 · Mon Jan 31 2022 22:10:25 GMT+0800 (China Standard Time)

port 9116 open? could it be that the log is generated due to the MIB Browser ?
try opening exporter_ip:9116 and manually fill in the required fields

@auterpring maybe you have 7.x version of rOS installed ? In version 7, delays in the response to SNMP on weak models were noticed

in Mikrotik log I can see, that exporter is working g

auterpring · Answer 10 · Mon Jan 31 2022 22:27:57 GMT+0800 (China Standard Time)

port 9116 open? could it be that the log is generated due to the MIB Browser ? try opening exporter_ip:9116 and manually fill in the required fields

@auterpring maybe you have 7.x version of rOS installed ? In version 7, delays in the response to SNMP on weak models were noticed

in Mikrotik log I can see, that exporter is working g

port 9116 is open

I resolve this issue using base snmp.yml from prometheus/snmp_exporter.
Now I try to generate new snmp.yml file

IgorKha · Answer 11 · Mon Jan 31 2022 23:57:12 GMT+0800 (China Standard Time)

@auterpring
The problem is not snmp.yml
Your problem is that Prometheus cannot reach port 9116 of the exporter
Try specifying a real (not localhost) exporter IP in prometheus.yml, like replacement: 192.168.1.5:9116

auterpring · Answer 12 · Tue Feb 01 2022 16:38:50 GMT+0800 (China Standard Time)

@auterpring The problem is not snmp.yml Your problem is that Prometheus cannot reach port 9116 of the exporter Try specifying a real (not localhost) exporter IP in prometheus.yml, like replacement: 192.168.1.5:9116

oh, it works with any other module

besides mikrotik