223 (and growing) Nginx rules to block bad bots.
Bad bots are defined as:
- E-mail harvesters
- Content scrapers
- Spam bots
- Vulnerability scanners
- Aggressive bots that provide little value
- Bots linked to viruses or malware
- Government surveillance bots
- Russian search engine Yandex
- Chinese search engine Baidu
Unless your website is written in Russian or Chinese, you probably don't get any traffic from them. They mostly just waste bandwidth and consume resources.
Bots try to make themselves look like other software by disguising their useragent. Their useragents may look harmless, perfectly legitimate even. For example, "^Java" but according to Project Honeypot, it's actually one of the most dangerous.
Include blacklist.conf on your nginx confg in http block : include /etc/nginx/blacklists.conf
In your server block, add :
if ($bad_bot) { return 403; } if ($bad_referer) { return 403; }