自用爬虫库
- 安装requirements.txt
pip install requirements.txt
- 使用scrapy命令
https://docs.scrapy.org/en/latest/intro/tutorial.html
- 爬虫使用案例示范:
scrapy crawl qianxun -o test.csv
scrapy crawl ranwen -o test.csv
笔趣岛(已失效)
scrapy crawl biqudao -o test.csv
爬取对应书籍
ipip.net
爬取IP段(例如Netflix Google Youtube)
# 爬https://whois.ipip.net/search/GOOGLE%20LLC
scrapy crawl ipip.net -o test.json -a keyword=GOOGLE%20LLC