Explicitly set the time zone in MySQL session

[Tore Anderson]

Description

When inserting data into a MySQL table, pmacct will issue queries such as:

UPDATE `acct_out` SET packets=packets+9, bytes=bytes+576, stamp_updated=NOW() WHERE FROM_UNIXTIME(1693526400) = stamp_inserted AND as_src=1042

I am using timestamps_utc: true and sql_history_roundoff: M here.

This UNIX timestamp corresponds to Fri Sep 1 00:00:00 UTC 2023, so the roundoff is working expected.

However, the MySQL time functions in use here – NOW() and FROM_UNIXTIME() – are sensitive to the MySQL server's time zone. If this is not set to UTC, it will result in stamp_updated and stamp_inserted being set to local time stamps instead of UTC ones (at least if you're using the DATETIME type suggested in the schemas found in the sql/ directory):

mysql> SHOW VARIABLES LIKE '%time_zone%';
+------------------+--------+
| Variable_name    | Value  |
+------------------+--------+
| system_time_zone | CEST   |
| time_zone        | SYSTEM |
+------------------+--------+
2 rows in set (0.00 sec)

mysql> SELECT FROM_UNIXTIME(1693526400);
+---------------------------+
| FROM_UNIXTIME(1693526400) |
+---------------------------+
| 2023-09-01 02:00:00       |
+---------------------------+
1 row in set (0.00 sec)

mysql> SELECT NOW();
+---------------------+
| NOW()               |
+---------------------+
| 2023-09-01 13:16:10 |
+---------------------+
1 row in set (0.00 sec)

This can be avoided by explicitly setting the session time zone:

mysql> SET time_zone = '+00:00';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT NOW();
+---------------------+
| NOW()               |
+---------------------+
| 2023-09-01 11:17:15 |
+---------------------+
1 row in set (0.01 sec)

mysql> SELECT FROM_UNIXTIME(1693526400);
+---------------------------+
| FROM_UNIXTIME(1693526400) |
+---------------------------+
| 2023-09-01 00:00:00       |
+---------------------------+
1 row in set (0.00 sec)

Setting the time zone must to be done on each (re)connection. It would be very useful if pmacct gained to capability to do so.

(It is of course possible to change the MySQL server's time zone, but that may require extra permissions and/or be undesirable because would have negative consequences for other applications/users.)

Version

sFlow Accounting Daemon, sfacctd 1.7.7-git [20211107-0 (ef37a415)]

Arguments:
 '--enable-mysql' '--enable-pgsql' '--enable-sqlite3' '--enable-kafka' '--enable-geoipv2' '--enable-jansson' '--enable-rabbitmq' '--enable-nflog' '--enable-ndpi' '--enable-zmq' '--enable-avro' '--enable-serdes' '--enable-redis' '--enable-gnutls' 'AVRO_CFLAGS=-I/usr/local/avro/include' 'AVRO_LIBS=-L/usr/local/avro/lib -lavro' '--enable-l2' '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' '--enable-st-bins'

Libs:
cdada 0.3.5
libpcap version 1.8.1
MariaDB 10.3.31
PostgreSQL 110013
sqlite3 3.27.2
rabbimq-c 0.11.0
rdkafka 1.8.2
jansson 2.14
MaxmindDB 1.6.0
ZeroMQ 4.3.2
Redis 1.0.3
GnuTLS 3.6.7
avro-c
serdes
nDPI 3.4.0
netfilter_log

Plugins:
memory
print
nfprobe
sfprobe
tee
mysql
postgresql
sqlite
amqp
kafka

System:
Linux 4.18.0-477.21.1.el8_8.x86_64 #1 SMP Tue Aug 8 21:30:09 UTC 2023 x86_64

Compiler:
gcc 8.3.0

For suggestions, critics, bugs, contact me: Paolo Lucente <[e-mail]>.

Appreciation
Done!

[Tore Anderson]

I found a way to handle this issue that does not require changes to pmacct nor admin privileges on the database server.

It is to use TIMESTAMP instead of DATETIME, changing the schema as follows:

-    stamp_inserted DATETIME NOT NULL,
-    stamp_updated DATETIME,
+    stamp_inserted TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    stamp_updated TIMESTAMP NULL,

This makes the database server store the timestamps as UTC internally, resolving all ambiguity.

[Paolo Lucente]

Hi Tore ( @toreanderson ),

It appears we were going ahead in parallel to resolve this, haha. I liked your original proposal, enforcing UTC timestamp on the current session. I have just implemented it with the commit above -- it appears to work good for me. I also see your alternative suggestion of using TIMESTAMP instead of DATETIME: i will find some room for it in the docs as an alternative.

Paolo