shubhsin / DMOZ-Scraper

Script I Created using BeautifulSoup to get all urls from dmoz.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

##DMOZ Scraper This is a scraper used to get all the URLs from DMOZ

It recursively goes into each URL to get URLs from them. In this way we have all the URLs from DMOZ.

##Usage

To run the code you need to have Python 2.7 installed on your machine. You will need package requests. To install this - pip install requests

You will need package BeautifulSoup. To install this - pip install beautifulsoup4

Once the installation is done, go to the directory in which the python file is present and run python scrapedmoz.py on your terminal Once it runs, just enter dmoz.org when prompted and you would see all the URLs in your terminal.

About

Script I Created using BeautifulSoup to get all urls from dmoz.org


Languages

Language:Python 100.0%