Follow links: true/false
eklem opened this issue · comments
Make it possible to not scrape content for links, and then not add to URL queue. Together with the possibility to add several URLs, this will make possible the use case where you know all the URLs you want the users to crawl up-front.