kvdaniel's repositories
tweeks
dummy
spiderfoot
SpiderFoot, the open source footprinting and intelligence-gathering tool.
pyspider
A Powerful Spider(Web Crawler) System in Python.
webmagic
A scalable web crawler framework.
pty.js
Bindings to forkpty(3) for node.js.
warehouse
Next Generation Python Package Repository
term.js
A terminal written in javascript.
crawlers
Some quick 'n dirty web crawlers.
retina-crawler
A news crawler for the Retina Project
node-crawler
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
testspiders
Useful test spiders for Scrapy
pySpidy
A simple, yet powerful, python web crawler for Google with browser capabilities
GoogleSearchCrawler
a tool for crawl Google search results
simDHT
A DHT crawler, very simple, writen by Python.
crawlera-tools
Crawlera tools
noskiddie
noskiddie is a basic log watcher to ban kids/scanner/crawler based on a blacklist
crawler4py
A web crawler in Python
web-crawlers
several py web crawlers
Zeek
Python distributed web scrapper and dynamic crawler
scrapy-spiders
Collection of python scripts I have created to crawl various websites, mostly for lead generation projects to match keywords and collect email addresses and post URLs
python-crawler-ccw
web resources crawler for pdf or doc by python 3
IWCT_Weibo_Crawler
This repository designing a sina weibo crawler is dedicated to the research program of IWCT,SJTU
brackets-open-in-cmder
Extension for Brackets to open the project folder in cmder.
brackets-terminal
Terminal Emulator for Brackets test editor
python-sitemap
Mini website crawler to make sitemap from a website.
yelpcrawl
Crawl and scrape Yelp's restaurant data for every zip code in the United States (or a specified zipcode). Yelp Crawler.
HtmlJsCrawler
Simple html and javascript files crawler