Giters

mitchellkrogza / nginx-ultimate-bad-bot-blocker

Nginx Block Bad Bots, Spam Referrer Blocker, Vulnerability Scanners, User-Agents, Malware, Adware, Ransomware, Malicious Sites, with anti-DDOS, Wordpress Theme Detector Blocking and Fail2Ban Jail for Repeat Offenders

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Integration with abuseipdb.com

robwent opened this issue · comments

commented

Hi!

I want to look into integrating the abuseipdb API.
My initial thought is to modify a script like this to create a new file of banned IPs that overwrites itself on a cron: https://github.com/ergin/nginx-cloudflare-real-ip

The question is where to put the file as it looks like each file in the bots.d directory is manually included from conf.d.

Should I just include it at the bottom of blacklist-ips.conf or is there a better way?

If I get it working, would you be interested in a pull request to integrate into the daily update script, or a separate similar script to update the file?

commented

I got this working:
https://github.com/robwent/abuseipdb-bad-bot-blocker

I get the feeling it's already integrated at some level as there are a lot of duplicate IPs after running it.

GitHub
Contribute to robwent/abuseipdb-bad-bot-blocker development by creating an account on GitHub.
commented

Hi @robwent abuse ipdb is already included in the daily updates but limited to the 1000 top IP's only without any API key or subscription. So this script you referenced, when used with a subscription to Abuse IPDB, would give you much more IP's but YES you will get many duplicate [warn] notices from Nginx but they are merely [warn] notices and are simply to be ignored as they do not affect Nginx operations in any way whatsoever.

commented

That's what I thought.

I started writing it with a free API key and that returns 10k IPs, all with a confidence level of 100%.

I then got a paid key and set it up on 2 servers at 75% confidence which returns approximately 77k results.
When I checked the logs this morning, I found that verified bingbot IPs had been blocked several times.
Their confidence level is in the 80s, so I've set confidence back to 90 and 63k results to err on the safe side.

My bash knowledge is nonexistent but running the script is quick and hardly uses any resources. I tried to add a bit of a fallback if there's no response from the API, but I think it will fail/cause issues if it returns anything other than the plain text list of IPs (As I found out when I requested the json response by mistake).

Hopefully, someone else will find it useful.