unclear to the user - freshclam --datadir only works with absolute paths and does not create directories recursively
TorstenEckern opened this issue · comments
Describe the bug
The user is never notified that freshclam --datadir
- does not create directories recursively
- needs an absolute path to run properly.
How to reproduce the problem
I'm using the default clamav 0.103.11 apt package installed on Ubuntu 20.04 (tag: focal-20221130 -> linux/amd64) in a fresh docker container:
#### Build Stage
################################################################################
ARG UBUNTU_BASE_TAG=focal-20221130
FROM ubuntu:${UBUNTU_BASE_TAG} as builder
ENV DEBIAN_FRONTEND=noninteractive
# Setup base system
# -----------------
RUN apt-get update && apt-get install -y \
clamav-freshclam \
&& rm -rf /var/lib/apt/lists/*
I run everything as root and directly after install of clamav-freshclam.
###########################
root@b5a47fb61fdc:/# freshclam -v --datadir=./foo/bar/clamav
Creating missing database directory: ./foo/bar/clamav
ERROR: Failed to create database directory: ./foo/bar/clamav
Manually prepare the database directory, or re-run freshclam with higher privileges.
ERROR: Initialization error!
Second experiment:
#############################
root@b5a47fb61fdc:/# freshclam -v --datadir=./clamav
Creating missing database directory: ./clamav
Assigned ownership of database directory to user "clamav".
Thu Feb 29 11:37:25 2024 -> *Current working dir is ./clamav/
Thu Feb 29 11:37:25 2024 -> *Can't open freshclam.dat in /sandbox/packages/m3d-virus-scan/clamav
Thu Feb 29 11:37:25 2024 -> *It probably doesn't exist yet. That's ok.
Thu Feb 29 11:37:25 2024 -> *Failed to load freshclam.dat; will create a new freshclam.dat
Thu Feb 29 11:37:25 2024 -> *Creating new freshclam.dat
Thu Feb 29 11:37:25 2024 -> *Saved freshclam.dat
Thu Feb 29 11:37:25 2024 -> ClamAV update process started at Thu Feb 29 11:37:25 2024
Thu Feb 29 11:37:25 2024 -> !Can't change dir to ./clamav/
Thu Feb 29 11:37:25 2024 -> *Querying current.cvd.clamav.net
Thu Feb 29 11:37:25 2024 -> *TTL: 1197
Thu Feb 29 11:37:25 2024 -> *fc_dns_query_update_info: Software version from DNS: 0.103.11
Thu Feb 29 11:37:25 2024 -> !Can't create temporary directory ./clamav/tmp.e41670c435
Thu Feb 29 11:37:25 2024 -> Hint: The database directory must be writable for UID 105 or GID 107
Thu Feb 29 11:37:25 2024 -> !Update failed.
That solved the first problem, but we're not yet where we want to be.
Apparently, directories are not being created recursively.
From the docs and command output, this is not clear, so I did not expect that.
This is the content of the newly created folder:
root@b5a47fb61fdc:/# ls -la clamav/
total 16
drwxr-xr-x 2 clamav clamav 4096 Feb 29 11:37 .
drwxrwxr-x 1 root root 4096 Feb 29 11:37 ..
-rw-r--r-- 1 clamav clamav 69 Feb 29 11:37 freshclam.dat
UID 105 and GID 107 belong to User and group clamav, respectively:
id -u clamav
105
id -g clamav
107
root@b5a47fb61fdc:/# cat /etc/clamav/freshclam.conf |grep Database
DatabaseOwner clamav
DatabaseDirectory /var/lib/clamav
DNSDatabaseInfo current.cvd.clamav.net
TestDatabases yes
CompressLocalDatabase no
DatabaseMirror db.local.clamav.net
DatabaseMirror database.clamav.net
Third experiment, using the defaults:
############################
root@b5a47fb61fdc:/# freshclam
Thu Feb 29 12:17:03 2024 -> ClamAV update process started at Thu Feb 29 12:17:03 2024
Thu Feb 29 12:17:03 2024 -> daily database available for download (remote version: 27200)
Time: 0.1s, ETA: 0.0s [========================>] 16B/16B
Thu Feb 29 12:17:03 2024 -> ^Can't download daily.cvd from https://database.clamav.net/daily.cvd
Thu Feb 29 12:17:03 2024 -> ^FreshClam received error code 429 from the ClamAV Content Delivery Network (CDN).
Thu Feb 29 12:17:03 2024 -> This means that you have been rate limited by the CDN.
Thu Feb 29 12:17:03 2024 -> 1. Run FreshClam no more than once an hour to check for updates.
Thu Feb 29 12:17:03 2024 -> FreshClam should check DNS first to see if an update is needed.
Thu Feb 29 12:17:03 2024 -> 2. If you have more than 10 hosts on your network attempting to download,
Thu Feb 29 12:17:03 2024 -> it is recommended that you set up a private mirror on your network using
Thu Feb 29 12:17:03 2024 -> cvdupdate (https://pypi.org/project/cvdupdate/) to save bandwidth on the
Thu Feb 29 12:17:03 2024 -> CDN and your own network.
Thu Feb 29 12:17:03 2024 -> 3. Please do not open a ticket asking for an exemption from the rate limit,
Thu Feb 29 12:17:03 2024 -> it will not be granted.
Thu Feb 29 12:17:03 2024 -> ^You are on cool-down until after: 2024-02-29 12:21:17
Thu Feb 29 12:17:03 2024 -> main database available for download (remote version: 62)
Time: 0.1s, ETA: 0.0s [========================>] 16B/16B
Thu Feb 29 12:17:03 2024 -> ^Can't download main.cvd from https://database.clamav.net/main.cvd
Thu Feb 29 12:17:03 2024 -> ^FreshClam received error code 429 from the ClamAV Content Delivery Network (CDN).
Thu Feb 29 12:17:03 2024 -> This means that you have been rate limited by the CDN.
Thu Feb 29 12:17:03 2024 -> 1. Run FreshClam no more than once an hour to check for updates.
Thu Feb 29 12:17:03 2024 -> FreshClam should check DNS first to see if an update is needed.
Thu Feb 29 12:17:03 2024 -> 2. If you have more than 10 hosts on your network attempting to download,
Thu Feb 29 12:17:03 2024 -> it is recommended that you set up a private mirror on your network using
Thu Feb 29 12:17:03 2024 -> cvdupdate (https://pypi.org/project/cvdupdate/) to save bandwidth on the
Thu Feb 29 12:17:03 2024 -> CDN and your own network.
Thu Feb 29 12:17:03 2024 -> 3. Please do not open a ticket asking for an exemption from the rate limit,
Thu Feb 29 12:17:03 2024 -> it will not be granted.
Thu Feb 29 12:17:03 2024 -> ^You are on cool-down until after: 2024-02-29 12:21:17
Thu Feb 29 12:17:03 2024 -> bytecode database available for download (remote version: 335)
Time: 0.2s, ETA: 0.0s [========================>] 282.94KiB/282.94KiB
Thu Feb 29 12:17:03 2024 -> Testing database: '/var/lib/clamav/tmp.f738edeacd/clamav-bc49313915fcc79dfa549a6a18db70ab.tmp-bytecode.cvd' ...
Thu Feb 29 12:17:03 2024 -> Database test passed.
Thu Feb 29 12:17:03 2024 -> bytecode.cvd updated (version: 335, sigs: 86, f-level: 90, builder: raynman)
Thu Feb 29 12:17:03 2024 -> !NotifyClamd: Can't find or parse configuration file /etc/clamav/clamd.conf
I'm on cooldown, but it does create the tmp directories! So why doesn't it do the same in my previous command?
Let's have a look at /var/lib/clamav:
root@b5a47fb61fdc:/# ls -la /var/lib/clamav/
total 300
drwxr-xr-x 1 clamav clamav 4096 Feb 29 12:17 .
drwxr-xr-x 1 root root 4096 Feb 28 12:18 ..
-rw-r--r-- 1 clamav clamav 289733 Feb 29 12:17 bytecode.cvd
-rw-r--r-- 1 clamav clamav 69 Feb 29 12:17 freshclam.dat
This looks better, too. Permissions and ownership are identical.
Now the 4th step, using an absolute path:
#############################
root@b5a47fb61fdc:/# freshclam --datadir=/clamav
Thu Feb 29 12:44:59 2024 -> ClamAV update process started at Thu Feb 29 12:44:59 2024
Thu Feb 29 12:44:59 2024 -> daily database available for download (remote version: 27200)
Time: 0.1s, ETA: 0.0s [========================>] 16B/16B
Thu Feb 29 12:44:59 2024 -> ^Can't download daily.cvd from https://database.clamav.net/daily.cvd
Thu Feb 29 12:44:59 2024 -> ^FreshClam received error code 429 from the ClamAV Content Delivery Network (CDN).
Thu Feb 29 12:44:59 2024 -> This means that you have been rate limited by the CDN.
Thu Feb 29 12:44:59 2024 -> 1. Run FreshClam no more than once an hour to check for updates.
Thu Feb 29 12:44:59 2024 -> FreshClam should check DNS first to see if an update is needed.
Thu Feb 29 12:44:59 2024 -> 2. If you have more than 10 hosts on your network attempting to download,
Thu Feb 29 12:44:59 2024 -> it is recommended that you set up a private mirror on your network using
Thu Feb 29 12:44:59 2024 -> cvdupdate (https://pypi.org/project/cvdupdate/) to save bandwidth on the
Thu Feb 29 12:44:59 2024 -> CDN and your own network.
Thu Feb 29 12:44:59 2024 -> 3. Please do not open a ticket asking for an exemption from the rate limit,
Thu Feb 29 12:44:59 2024 -> it will not be granted.
Thu Feb 29 12:44:59 2024 -> ^You are on cool-down until after: 2024-03-01 12:44:59
Thu Feb 29 12:44:59 2024 -> main database available for download (remote version: 62)
Time: 0.1s, ETA: 0.0s [========================>] 16B/16B
Thu Feb 29 12:44:59 2024 -> ^Can't download main.cvd from https://database.clamav.net/main.cvd
Thu Feb 29 12:44:59 2024 -> ^FreshClam received error code 429 from the ClamAV Content Delivery Network (CDN).
Thu Feb 29 12:44:59 2024 -> This means that you have been rate limited by the CDN.
Thu Feb 29 12:44:59 2024 -> 1. Run FreshClam no more than once an hour to check for updates.
Thu Feb 29 12:44:59 2024 -> FreshClam should check DNS first to see if an update is needed.
Thu Feb 29 12:44:59 2024 -> 2. If you have more than 10 hosts on your network attempting to download,
Thu Feb 29 12:44:59 2024 -> it is recommended that you set up a private mirror on your network using
Thu Feb 29 12:44:59 2024 -> cvdupdate (https://pypi.org/project/cvdupdate/) to save bandwidth on the
Thu Feb 29 12:44:59 2024 -> CDN and your own network.
Thu Feb 29 12:44:59 2024 -> 3. Please do not open a ticket asking for an exemption from the rate limit,
Thu Feb 29 12:44:59 2024 -> it will not be granted.
Thu Feb 29 12:44:59 2024 -> ^You are on cool-down until after: 2024-03-01 12:44:59
Thu Feb 29 12:44:59 2024 -> bytecode database available for download (remote version: 335)
Time: 0.1s, ETA: 0.0s [========================>] 282.94KiB/282.94KiB
Thu Feb 29 12:45:00 2024 -> Testing database: '/clamav/tmp.dadc52d396/clamav-ffe7d69729e5e531fc5a17487a937d23.tmp-bytecode.cvd' ...
Thu Feb 29 12:45:00 2024 -> Database test passed.
Thu Feb 29 12:45:00 2024 -> bytecode.cvd updated (version: 335, sigs: 86, f-level: 90, builder: raynman)
Thu Feb 29 12:45:00 2024 -> !NotifyClamd: Can't find or parse configuration file /etc/clamav/clamd.conf
This works!!
Take home message:
Please notify the user that freshclam --datadir
- does not create directories recursively
- needs an absolute path
This cost me 1 day to find out.
Thanks a lot for appreciating my effort ;)
Hi,
Thank you for putting the time in. I'll put in a ticket internally to create the directories and update the help messages for our next release.
Thanks,
Andy
Hi,
The clamav team has decided that it is best to let the user create the directories with the permissions that they require, so freshclam will not be creating the directories for 'datadir'. We have updated the usage help and man pages to make this more clear to the user. Thank you for your ticket, the update will be in our next release.
Thanks,
Andy