JayBizzle / Crawler-Detect

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

Home Page:https://crawlerdetect.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Agent with '+http...' almost always a bot.

bartonlp opened this issue · comments

Any time you find a user agent with '+http' it is almost always a bot. So for example, '+https://help.something.com' will probably be a bot. The 'good' bots almost always have a place where you can see who they are.

"Probably" is a bit loose for my liking, unless you can convince me otherwise 🤔