berlinbrown / octanewebcrawler-data

Web Crawler Data and Analysis

README

---

This project contains data collected from the web crawler project.

Download seed files or minimal set of URL files.

Use the seed file in your web crawler projects.