Whitelist not being applied
jasonwryan opened this issue · comments
Under 0.999.3-1 the whitelist is not applied to the newly processed hosts file.
To reproduce, use hostsblock-urlcheck to explicity add a URL to the white list (eg, http://www.lrb.co.uk/) and then run hostsblock. The newly generated /etc/hosts file will include the whitelisted URL.
To debug, I appended a message to the relevant line in /usr/bin/hostsblock, like so
sed "s/ \!.*$//g" | sort -u | grep -vf "$whitelist" >> "$hostsfile" && _notify 4 "Whitelist applied +++++"; then
This never gets printed.
The output of hostsblock (with level 4 debugging) is:
[INFO] Old /etc/hosts will not be recycled into new version.
[DETAIL] Backing up old version of /etc/hosts...
/usr/lib/hostsblock-common.sh: line 55: 0: command not found
[WARN] FAILED to compress /etc/hosts with 0.
[DETAIL] Using a hostshead file, so overwriting /etc/hosts with /etc/hostsblock/hosts.head...
[INFO] Replaced existing /etc/hosts with .
[INFO] Compiling block entries into /etc/hosts...
[INFO] Compiled block entries into /etc/hosts.
[DETAIL] Skipping redirect entries...
[INFO] Appending blacklisted entries to /etc/hosts...
[INFO] Appended blacklisted entries to /etc/hosts.
[INFO] /etc/hosts: 354005 urls redirected to 127.0.0.1.
[INFO] Executing postprocessing...
[INFO] Postprocessing completed.
[DETAIL] Cleaning up temporary directory /dev/shm/hostsblock...
[NOTE] Cleaned up /dev/shm/hostsblock.
[INFO] DONE.
The error message about 0 not found points to /etc/hostsblock/hostblock.conf where I have tried both 0 and no here:
backup_old="0"
Sorry I haven't been able to debug more...
I am affected by the same bug. Also hostsblock is totally ignoring my blocklist in /etc/hostsblock/hostsblock.conf and even loads commented lists.
I looked into this some more. The whitelist is only applied if redirects is set to 1 (?) and then rather than whitelisted sites being excluded from /etc/hosts, they are included.
I use the same version of hostsblock on Ubuntu 15.10 with this indicator of mine: https://github.com/Sadi58/hostsblock (only added an "indiractor" folder)
I have "Allow Redirects" option intact, i.e. default value of "0"
My whitelist works as it should. For instance I've added "clients2.google.com" as well as its 3,4&5 versions, as can be seen in "hostsfile.mine.nu.Hosts" blocklist, and they are filtered out.
The same goes for the blacklist as well.
PS: Meanwhile I've just noticed that some entries in the current blacklist are no longer necessary as they are now included in the blocklists used.
Hmm. I haven't been able to reproduce this with redirects turned off either, like @Sadi58.
@jasonwryan, @marhkb, could you post excerpts from your "hosts.block" or "hosts" files and possibly also something from your whitelist? I might know where the issues with the commented lines come from, but I don't immediately see how that interrelates to the inverted white list issue.
My issues:
- Blocklist selection ignored
backup_old="0"ignored (hosts.block.oldis created).
However, blacklist and whitelist are working.
/etc/hostsblock/white.list:
000007.ru
/etc/hostsblock/black.list:
blacklisttest.com
/etc/hosts.block:
127.0.0.1 localhost.localdomain localhost
::1 localhost.localdomain localhost
# End of file
[Here would be 000007.ru if not whitelisted]
127.0.0.1 0000663c.tslocosumo.us
127.0.0.1 0000a-fast-proxy.de
[...]
127.0.0.1 zzzzzz.com
127.0.0.1 blacklisttest.com
Config changes:
hostsfile="/etc/hosts.block" # For use with a dnscacher, e.g dnsmasq
#hostsfile="/etc/hosts" # DEFAULT. If not using a dns caching daemon
[...]
# backup_old="gzip" # DEFAULT
backup_old="0"
[...]
#recycle_old=1 # DEFAULT
recycle_old="0"
For testing:
- Disabled all blocklists except http://winhelp2002.mvps.org/hosts.zip
- Restarted dnsmasq and hostsblock
- Ran
hostsblock -u -v 3
Output:
[INFO] Checking blocklists for updates...
[INFO] Refreshed blocklist http://winhelp2002.mvps.org/hosts.zip.
[INFO] Changes found among blocklists. Extracting and preparing cached files to working directory...
[INFO] Unzipped hostsfile.mine.nu.Hosts.zip.
[INFO] Unzipped hostsfile.org.Downloads.BadHosts.unx.zip.
[INFO] Unzipped hosts-file.net.download.hosts.zip.
[INFO] Unzipped winhelp2002.mvps.org.hosts.zip.
[INFO] Old /etc/hosts.block will not be recycled into new version.
/usr/lib/hostsblock-common.sh: line 55: 0: command not found
[WARN] FAILED to compress /etc/hosts.block with 0.
[INFO] Replaced existing /etc/hosts.block with .
[INFO] Compiling block entries into /etc/hosts.block...
[INFO] Compiled block entries into /etc/hosts.block.
[INFO] Appending blacklisted entries to /etc/hosts.block...
[INFO] Appended blacklisted entries to /etc/hosts.block.
[INFO] /etc/hosts.block: 456417 urls redirected to 127.0.0.1.
[INFO] /etc/hosts.block: 1 urls redirected to 127.0.0.1 localhost.localdomain localhost.
[INFO] Executing postprocessing...
[INFO] Postprocessing completed.
[NOTE] Cleaned up /dev/shm/hostsblock.
[INFO] DONE.
Issues from the log:
- Redirects 456417 URLs, the same number as when all blocklists are disabled
- Tries to compress with "0"?
- "Replaced existing /etc/hosts.block with ." With?
| User | Whitelist | Blocklist selection | backup_old |
|---|---|---|---|
| jasonwryan | - | + | - |
| marhkb | - | - | ? |
| Markus00000 | + | - | - |