The package is not detecting Ahrefs as a crawler

Question

The package is not detecting Ahrefs as a crawler

mkrvaric opened this issue a year ago · comments

This line from the access log, is not detected as a crawler. Why Ahrefs bot is not included?

54.36.149.243 - - [29/Jun/2023:08:10:28 +0000] "GET /hr/proizvod/tajice-nike-w-pro-365-tight-35524 HTTP/2.0" 200 16858 "-" "Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit)"

Mark Beech · Answer 1 · Thu Jul 06 2023 04:48:21 GMT+0800 (China Standard Time)

Works for me https://crawlerdetect.io/?q=Mozilla/5.0%20(compatible;%20AhrefsSiteAudit/6.1;%20+http://ahrefs.com/robot/site-audit)