not working on somewebsites
rootThatBox opened this issue · comments
"/go/bin/hakrawler -plain -url https://ads.pinterest.com"
only finds 1 endpoint ie robots.txt visiting manually we can see that there are many other endpoints and urls .
Output:
https://ads.pinterest.com/
hakrawler version:beta11
go version :go1.15.5 linux/amd64
Fixed in latest version