rahulr92 / justdialscrapper

Scrapper and D3 visualization

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

#Python Scraper and D3 visualization using Play 1.2.5

This is a Play 1.2.5 app and the scraper written in python resides in app/controllers/JustDialScrap.py

TO RUN SCRAPER

repohome#>cd app/controllers/

repohome#>python JustDialScrap.py

City: User Input I

Category: User Input II

Example:

repohome#>python JustDialScrap.py

city: chennai

Category: cabs

Above commands runs the scraper and puts the scraped data to Chennai-Cabs.csv file and also generates Chennai-Cabs.json file which has the json data from JustDials top 5 pages(max) verified vs nonverified count for the city Chennai and category cabs.

TO RUN PLAY

repohome#> path to play 1.2.5/play run

which runs the webapp with default play dev settings on port 9000.

TO VIEW THE CHART open broswer url: localhost:9000

Json files generated by scraper is read by Play controller(app/controllers/Application.java) and then passed via Groovy templating to D3 chart scripts which is inside /app/views/Application/index.html. The play app uses D3 Pie-Chart to visualize the verified vs nonverified count.

About

Scrapper and D3 visualization


Languages

Language:Python 60.3%Language:Java 35.6%Language:PHP 3.4%Language:Shell 0.7%