Giters

rfmoz / tuptime

Report historical and statistical real time of the system, keeping it between restarts. Like uptime command but with more interesting output.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

tuptime shows wrong donwtime

kumaresan078 opened this issue · comments

commented

Hi All,

Good Day!

The tuptime shows wrong downtime around 1hr 21 min while the server up within minutes as per audit logs.Could someone help on this?

No. Startup T. Uptime Shutdown T. End Downtime

1 03:11:16 PM 03/12/2022 46d 21h 44m 49s 12:56:05 PM 04/28/2022 BAD 1h 21m 10s
2 02:17:15 PM 04/28/2022 22h 33m 59s

Audit logs:

type=SYSTEM_SHUTDOWN msg=audit(04/28/2022 14:16:52.959:968962) : pid=189010 uid=root auid=unset ses=unset msg=' comm=systemd-update-utmp exe=/usr/lib/systemd/systemd-update-utmp hostname=? addr=? terminal=? res=success'

type=SYSTEM_BOOT msg=audit(04/28/2022 14:17:25.947:168) : pid=1238 uid=root auid=unset ses=unset msg=' comm=systemd-update-utmp exe=/usr/lib/systemd/systemd-update-utmp hostname=? addr=? terminal=? res=success'

Regards
Kumar

commented

Hi Kumar,

To register the times properly, tuptime requires an execution at startup, shutdown and optionally (but highly recommended) at regular intervals with cron, by default each 5 minutes. With the last one, in case of power failure, you only lost 5 o less minutes, but in your case is more than an hour.

It seems that it wasn't executed properly at shutdown and maybe also miss the cron execution.

How did you install tuptime?

Could you check if the output of the following commands confirm the right executions at startup, by cron, and at shutdown?
$ systemctl status tuptime
$ cat /etc/cron.d/tuptime
$ journalctl -u tuptime
$ journalctl -u cron | grep tuptime

Regards,

commented

Hi,

Please find below update.

Could you confirm both cron & systemd is mandatory or either one sufficient?

Not sure why the shutdown registered on April 28th 12:56 while the actual reboot taken place at 14:16

Startup: 1 at 03:11:16 PM 03/12/2022
Uptime: 46d 21h 44m 49s
Shutdown: BAD at 12:56:05 PM 04/28/2022
Downtime: 1h 21m 10s

Startup: 2 at 02:17:15 PM 04/28/2022
Uptime: 1d 6h 15m 51s

On April 28th around 12:48 installed tuptime manually on the server thought it sufficient.

cp /tmp/tuptime/src/tuptime /usr/bin/tuptime
chmod ugo+x /usr/bin/tuptime

On April 28th around 14:16 the server got rebooted later it up by 14:17

On April 28th 14:34 installed tuptime again with cron & services.

install -m 755 /tmp/tuptime/src/tuptime /usr/bin/tuptime
chmod ugo+x /usr/bin/tuptime
useradd --system --no-create-home --home-dir '/var/lib/tuptime' --shell '/bin/false' --comment 'Tuptime execution user' _tuptime
groupadd _tuptime
tuptime
chown -R _tuptime:_tuptime /var/lib/tuptime
install -m 644 /tmp/tuptime/src/cron.d/tuptime /etc/cron.d/tuptime
install -m 644 /tmp/tuptime/src/systemd/tuptime.service /usr/lib/systemd/system/tuptime.service
systemctl enable tuptime.service && systemctl start tuptime.service

Output:

$ systemctl status tuptime

● tuptime.service - Tuptime service
Loaded: loaded (/usr/lib/systemd/system/tuptime.service; enabled; vendor preset: disabled)
Active: active (exited) since Thu 2022-04-28 22:07:54 +08; 23h ago
Docs: man:tuptime(1)
file:///usr/share/doc/tuptime/tuptime-manual.txt.gz
Process: 221674 ExecStart=/usr/bin/tuptime -x (code=exited, status=0/SUCCESS)
Main PID: 221674 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/tuptime.service

Apr 28 22:07:54 server1 systemd[1]: Starting Tuptime service...
Apr 28 22:07:54 server1 systemd[1]: Started Tuptime service.

$ cat /etc/cron.d/tuptime

/etc/cron.d/tuptime: crontab entry for tuptime update.

NOTE: Decrease the execution time for increase accuracity.

*/5 * * * * _tuptime if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi

$ journalctl -u tuptime

-- Logs begin at Mon 2022-04-25 20:03:31 +08, end at Fri 2022-04-29 21:25:18 +08. --
Apr 28 22:07:54 server1 systemd[1]: Starting Tuptime service...
Apr 28 22:07:54 server1 systemd[1]: Started Tuptime service.

$ journalctl -u cron | grep tuptime

No output

Regards
Kumar

commented

Due the incomplete installation at the beginning, the database file doesn't have the right information.

Please, remove the file /var/lib/tuptime/tuptime.db, restart the tuptime service and set the right permissions chown -R _tuptime:_tuptime /var/lib/tuptime. Just trust only after this momment.

Also, check why cron isn't executed the script located in /etc/cron.d/tuptime.

commented

Hi Rfrail,

I did below steps initially on most of the servers later performed complete steps.It means all servers will have similar issue?

Could you confirm both cron & systemd is mandatory or either one sufficient?

cp /tmp/tuptime/src/tuptime /usr/bin/tuptime
chmod ugo+x /usr/bin/tuptime

Later:

install -m 755 /tmp/tuptime/src/tuptime /usr/bin/tuptime
chmod ugo+x /usr/bin/tuptime
useradd --system --no-create-home --home-dir '/var/lib/tuptime' --shell '/bin/false' --comment 'Tuptime execution user' _tuptime
groupadd _tuptime
tuptime
chown -R _tuptime:_tuptime /var/lib/tuptime
install -m 644 /tmp/tuptime/src/cron.d/tuptime /etc/cron.d/tuptime
install -m 644 /tmp/tuptime/src/systemd/tuptime.service /usr/lib/systemd/system/tuptime.service
systemctl enable tuptime.service && systemctl start tuptime.service

Cron logs: As per below logs it being executed however in journal log it doesn't show up.

Apr 29 21:50:01 server1 CROND[25198]: (_tuptime) CMD ( if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi)
Apr 29 21:55:01 server1 CROND[27064]: (_tuptime) CMD ( if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi)
Apr 29 22:00:01 server1 CROND[29058]: (_tuptime) CMD ( if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi)
Apr 29 22:05:01 server1 CROND[30939]: (_tuptime) CMD ( if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi)
Apr 29 22:10:01 server1 CROND[32863]: (_tuptime) CMD ( if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi)
Apr 29 22:15:01 server1 CROND[34833]: (_tuptime) CMD ( if [ -x /usr/bin/tuptime ]; then /usr/bin/tuptime -x > /dev/null; fi)

Regards
Kumar

commented

About the installation, why don't you use the package manager from your Linux distribution? A simple apt install tuptime or dnf install tuptime surely help you to avoid doing it by hand.

Systemd and Cron are necessary. If you miss one, you're in risk to lost information.

commented

Hi Rfrail3,

The reason for manual installation is we are using different flavors of Linux. The most we use suse linux which i don't find support here.

As per below instruction thought either one cron or systemd sufficient.

Copy cron file:

       # install -m 644 tuptime/src/cron.d/tuptime /etc/cron.d/tuptime

   If it use systemd, copy service file and enable it:

       # install -m 644 tuptime/src/systemd/tuptime.service /lib/systemd/system/tuptime.service
       # systemctl enable tuptime.service && systemctl start tuptime.service

Regards
Kumaresan

commented

Hi Rfrail3,

Could you provide update on this? Is the data not accurate incase of either systemd service or cron not running?

Do we need to monitor tuptime service?

Regards
Kumar

commented

The time isn't accurete If the systemd service isn't running? absolutely yes

The time isn't accurete If the cron service isn't running? only in case that the system have a force shutdown, but no one knows when it will happen, so yes.

Tuptime isn't a service resident on memory, it only executes (and exit) by systemd at startup and shutdown, so there aren't anything to monitor along the time that the system is running.

commented

Thanks Rfrail3.Appreciate your help.

commented

Thanks Rfrail3.Appreciate your help.