dongweiming / daenerys

Scraping and Web Crawling Framework For Zhihu Live

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Getting started

git clone https://github.com/dongweiming/daeneryscd daenerysvirtualenv venvsource venv/bin/activatepip install -r requirements.txt
# 安装配置Redis和MongoDBcd demopython worker.py # 启动worker(默认5个进程)python beat.py  # 新开启一个终端,启动Beat服务生成任务ipython  # 发布一个需要获取执行结果的任务
In [1]: from messaging import sync_get

In [2]: sync_get('flask')
PUT flask
Out[2]:
{u'author': u'Armin Ronacher',
 u'download_url': u'https://pypi.python.org/packages/24/6e/11b9c57e46f276a8a8da85a2fa7ada62b0463b68693616c7ab5df356fa/Flask-0.12.1.tar.gz',
 u'name': u'flask',
 u'version': u'0.12.1'}

About

Scraping and Web Crawling Framework For Zhihu Live

License:Apache License 2.0


Languages

Language:Python 100.0%