Giters

pmacct / pmacct

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

Home Page:http://www.pmacct.net

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

pmacct 1.7.8 - no such column: vlan_in

JonMurphy opened this issue · comments

commented

Description
What's the issue? Did something go wrong? Would you like something new to exist?

pmacct stopped placing data into sqlite. I see lots of these errors:

Jun 25 13:45:01 ipfire pmacctd[24294]: INFO ( plugin3/sqlite3 ): *** Purging cache - START (PID: 24294) *** 
Jun 25 13:45:01 ipfire pmacctd[24294]: ERROR ( plugin3/sqlite3 ): no such column: vlan_in  
Jun 25 13:45:01 ipfire pmacctd[24294]: INFO ( plugin3/sqlite3 ): *** Purging cache - END (PID: 24294, QN: 0/226, ET: 0) ***

This happen after the upgrade of pmacct from from version 1.7.6 to 1.7.8.

I do not have vlan_in in my pmacct.conf file. And no vlan_in in sqlite.

I turned on debug in the conf file and now see this in the logs:

Jun 25 15:30:01 ipfire pmacctd[1776]: INFO ( plugin3/sqlite3 ): *** Purging cache - START (PID: 1776) *** 
Jun 25 15:30:01 ipfire pmacctd[1776]: DEBUG ( plugin3/sqlite3 ): FAILED query follows: UPDATE acct SET packets=packets+797, bytes=bytes+171373, stamp_updated=DATETIME('now', 'localtime') WHERE DATETIME(1687669200, 'unixepoch', 'localtime') = stamp_inserted AND mac_src='nn:nn:b9:5a:b9:db' AND mac_dst='nn:nn:5e:00:00:fb' AND vlan_in=0 AND ip_src='192.168.60.1' AND ip_dst='224.0.0.251' AND src_port=0 AND dst_port=0 AND ip_proto='0' 
Jun 25 15:30:01 ipfire pmacctd[1776]: ERROR ( plugin3/sqlite3 ): no such column: vlan_in  
Jun 25 15:30:01 ipfire pmacctd[1776]: DEBUG ( plugin3/sqlite3 ): FAILED query follows: UPDATE acct SET packets=packets+217, bytes=bytes+85483, stamp_updated=DATETIME('now', 'localtime') WHERE DATETIME(1687669200, 'unixepoch', 'localtime') = stamp_inserted AND mac_src='nn:nn:b9:5a:b9:db' AND mac_dst='nn:nn:86:42:f8:10' AND vlan_in=0 AND ip_src='nn.nn.27.38' AND ip_dst='192.168.60.212' AND src_port=0 AND dst_port=0 AND ip_proto='0' 
Jun 25 15:30:01 ipfire pmacctd[1776]: ERROR ( plugin3/sqlite3 ): no such column: vlan_in  
Jun 25 15:30:01 ipfire pmacctd[1776]: DEBUG ( plugin3/sqlite3 ): FAILED query follows: UPDATE acct SET packets=packets+256, bytes=bytes+44502, stamp_updated=DATETIME('now', 'localtime') WHERE DATETIME(1687669200, 'unixepoch', 'localtime') = stamp_inserted AND mac_src='nn:nn:86:42:f8:10' AND mac_dst='nn:nn:b9:5a:b9:db' AND vlan_in=0 AND ip_src='192.168.60.212' AND ip_dst='nn.nn.27.38' AND src_port=0 AND dst_port=0 AND ip_proto='0' 
Jun 25 15:30:01 ipfire pmacctd[1776]: ERROR ( plugin3/sqlite3 ): no such column: vlan_in

lots of vlan_in.

IMHO - this seems similar to: #695

Version

[root@ipfire ~] # pmacct -V
pmacct IMT plugin client, pmacct 1.7.8-git (RELEASE)

Appreciation
Please consider starring this project to boost our reach on github!
already done!

commented

Hi @JonMurphy ,

Thanks for reporting this issue, i agree this seems the same issue as the other case opened. Just in case, can you also please share your config?

Paolo

commented

here ya go!

!
! Pmacctd configuration file for IPFire environment
!

!-----------------------------------  global  --------------------------------------------

syslog: daemon
daemonize: true
!debug: false
debug: true
promisc: true
!interface: green0
pcap_interface: green0

imt_mem_pools_number: 256

plugins: memory[plugin1], sqlite3[plugin3]


!-----------------------------------  memory  --------------------------------------------

!
! "plugin1" plugin configuration
!
!plugins: memory[plugin1]

plugin_buffer_size[plugin1]: 102400
plugin_pipe_size[plugin1]:   102400000

imt_path[plugin1]: /var/spool/pmacct/plugin1.pipe

aggregate[plugin1]: src_host, src_mac, dst_host, dst_mac
aggregate_filter[plugin1]: ip

!-----------------------------------  sqlite3  -------------------------------------------

!
! "plugin3" plugin configuration
!
!plugins: sqlite3[plugin3]
!	small DB - 4 million row/line - 9.7 MB
!	since 2021-05-20
!   save once day
!	refresh save every 15 minutes / 900 sec
!	ignore LAN-to-LAN

plugin_buffer_size[plugin3]: 102400
plugin_pipe_size[plugin3]: 10240000

sql_db[plugin3]: /var/spool/pmacct/pmacct_sqlitev1p3_copy.db
sql_table_version[plugin3]: 1
sql_history[plugin3]: 1d
sql_refresh_time[plugin3]: 900
sql_history_roundoff[plugin3]: h

aggregate[plugin3]: src_host, src_mac, dst_host, dst_mac
aggregate_filter[plugin3]: ip and not (dst and src net 192.168.0.0/16)
commented

hi Jon ( @JonMurphy ), this was a bug should now be resolved with the latest commit. If you get a chance, let me know if this works for you. Paolo

commented

Paolo - thank you for looking into this and fixing so quickly!
After this is tested, when might this be built into the next version (e.g., v1.7.9)? A guess is fine! (sorry for asking and I promise not to hold you to the guesstimate!) Jon

commented

Yes, this will be part of 1.7.9. There is no committed date yet for its release but for sure it will be by the end of the year, with code freeze & version bump already taking place in a couple of months. This said, please start having a test to current code so to see all is working good for you. Keep me posted. Paolo

commented

hi Jon ( @JonMurphy ), this was a bug should now be resolved with the latest commit. If you get a chance, let me know if this works for you. Paolo

Sorry I forgot to close the loop on this! This was tested on 2023-07-11 and 2023-07-12. After a day of running, all seems A-OK! no vlan errors

THANK YOU!