A simple class for parsing robots.txt files and telling whether certain paths are allowed for certain user agents
Geek Repo:Geek Repo
Github PK Tool:Github PK Tool
