Aggregate MRTG graphs are including the reseller uplink interfaces as well as the peering interfaces, creating incorrect graphs/stats.
ichilton opened this issue · comments
ISSUE TYPE
Bug Report
OS
VERSION
define( 'APPLICATION_VERSION', '6.3.0' );
define( 'APPLICATION_VERDATE', '2022110200' );
(latest on release-v6 branch)
ENVIRONMENT
PHP 8.0.14 (cli) (built: Dec 20 2021 21:22:57) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.14, Copyright (c) Zend Technologies
with Zend OPcache v8.0.14, Copyright (c), by Zend Technologies
ichilton@portal:~$ dpkg -l | grep php
ii libapache2-mod-php8.0 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 server-side, HTML-embedded scripting language (Apache 2 module)
ii php-common 2:91+ubuntu20.04.1+deb.sury.org+1 all Common files for PHP packages
ii php-composer-ca-bundle 1.2.6-1 all utility library to find a path to the system CA bundle
ii php-composer-semver 1.5.1-1 all utilities, version constraint parsing and validation
ii php-composer-spdx-licenses 1.5.3-1 all SPDX licenses list and validation library
ii php-composer-xdebug-handler 1.4.0-1 all Restarts a process without Xdebug
ii php-json-schema 5.2.9-1 all implementation of JSON schema
ii php-psr-container 1.0.0-2 all Common Container Interface (PHP FIG PSR-11)
ii php-psr-log 1.1.2-1 all common interface for logging libraries
ii php-symfony-console 4.3.8+dfsg-1ubuntu1 all run tasks from the command line
ii php-symfony-filesystem 4.3.8+dfsg-1ubuntu1 all basic filesystem utilities
ii php-symfony-finder 4.3.8+dfsg-1ubuntu1 all find files and directories
ii php-symfony-process 4.3.8+dfsg-1ubuntu1 all execute commands in sub-processes
ii php-symfony-service-contracts 1.1.8-1 all Generic abstractions related to writing services
ii php8.0 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 all server-side, HTML-embedded scripting language (metapackage)
ii php8.0-bcmath 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 Bcmath module for PHP
ii php8.0-cgi 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 server-side, HTML-embedded scripting language (CGI binary)
ii php8.0-cli 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 command-line interpreter for the PHP scripting language
ii php8.0-common 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 documentation, examples and common module for PHP
ii php8.0-curl 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 CURL module for PHP
ii php8.0-ds 1.4.0-1+ubuntu20.04.1+deb.sury.org+1 amd64 PHP extension providing efficient data structures for PHP 7
ii php8.0-fpm 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 server-side, HTML-embedded scripting language (FPM-CGI binary)
ii php8.0-gd 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 GD module for PHP
ii php8.0-igbinary 3.2.6+2.0.8-2+ubuntu20.04.1+deb.sury.org+3 amd64 igbinary PHP serializer
ii php8.0-intl 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 Internationalisation module for PHP
ii php8.0-mbstring 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 MBSTRING module for PHP
ii php8.0-memcache 8.0+4.0.5.2+3.0.9~20170802.e702b5f9+-3+ubuntu20.04.1+deb.sury.org+3 amd64 memcache extension module for PHP
ii php8.0-memcached 3.1.5+2.2.0-10+ubuntu20.04.1+deb.sury.org+3 amd64 memcached extension module for PHP, uses libmemcached
ii php8.0-msgpack 2.2.0~rc1+2.1.2+0.5.7-2+ubuntu20.04.1+deb.sury.org+3 amd64 PHP extension for interfacing with MessagePack
ii php8.0-mysql 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 MySQL module for PHP
ii php8.0-opcache 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 Zend OpCache module for PHP
ii php8.0-readline 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 readline module for PHP
ii php8.0-rrd 2.0.3+1.1.3-2+ubuntu20.04.1+deb.sury.org+3 amd64 PHP bindings to rrd tool system
ii php8.0-snmp 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 SNMP module for PHP
ii php8.0-xml 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 DOM, SimpleXML, XML, and XSL module for PHP
ii php8.0-yaml 2.2.2+2.1.0+2.0.4+1.3.2-1+ubuntu20.04.1+deb.sury.org+3 amd64 YAML-1.1 parser and emitter for PHP
ii php8.0-zip 8.0.14-1+ubuntu20.04.1+deb.sury.org+1 amd64 Zip module for PHP
CONFIGURATION
Don't want to put that on GitHub.
SUMMARY
We are in the process of moving from physical loopback cables to EOS layer 2 subinterfaces to provide reseller ports.
We have just his morning moved our first which is doing a significant amount of traffic (~50G) as it's actually an ASN share rather than reseller.
We are now seeing that our traffic graphs have ~40G difference between IN and OUT traffic.
I had a hunch what this would be and have confirmed it - the MRTG aggregate graphs are graphing the reseller uplink interfaces as well as the peering interfaces.
i.e it's graphing both the traffic on the physical port as well as the subinterfaces.
I have verified this by manually editing mrtg.cfg and removing each occurrence of that physical port and the IN and OUT have now converged to within a few hundred Mbps, which is about right for the other resellers.
It should just be including the 'peering' ports, not those set to 'reseller'.
IMPORTANCE
HIGH - this is affecting our production traffic graphs.
is the reseller NNI physical interface set to type: Reseller
?
@ichilton - I'll need blow away some cobwebs to look at that code - hasn't been changed in years and it seems slightly odd that this hasn't come up before. E.g. if this is the cause, graphing the reseller port would be done in both fan out mode and veth hairpinning mode, I think.
(edited to remove reference to disabling polling - that's per-switch, not per-interface)
is the reseller NNI physical interface set to type:
Reseller
?
Yep!
...and the two sub-interfaces set to 'Peering'.
@ichilton - I'll need blow away some cobwebs to look at that code - hasn't been changed in years and it seems slightly odd that this hasn't come up before.
As I say, we only noticed it because this 'reseller' is doing more than a trivial amount of traffic (40Gbps in this case).
Happy to send screenshots or screen share demo on a zoom call if you think we might have something slightly different to you / wrong, rather than the code?
...but the parent interfaces are definitely set to type 'Reseller' and are appearing in the MRTG aggregated graphs.
Thanks,
Ian
Fixed via e40be75