LxL's repositories

scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:

Language:PythonLicense:GPL-3.0Stargazers:3034Issues:75Issues:181

files

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

scrapyd-cluster-on-heroku

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO :point_right:

Language:PythonLicense:GPL-3.0Stargazers:121Issues:7Issues:12

logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

Language:PythonLicense:GPL-3.0Stargazers:86Issues:5Issues:10

scrapyd

[PR #326] Native support for basic auth :lock: `pip install -U git+https://github.com/scrapy/scrapyd.git`, then add `username = yourusername` and `password = yourpassword` in the scrapyd.conf file. DOCS :point_right:

Language:PythonLicense:BSD-3-ClauseStargazers:10Issues:3Issues:0

scrapyd-cluster-on-heroku-scrapyd-app

How to set up Scrapyd cluster on Heroku

Language:PythonLicense:GPL-3.0Stargazers:9Issues:3Issues:1

awesome-scrapy

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

notes

Keep on reading

Language:PythonStargazers:6Issues:3Issues:0

awesome-python-cn

Python资源大全中文版,包括:Web框架、网络爬虫、模板引擎、数据库、数据可视化、图片处理等,由伯乐在线持续更新。

Language:MakefileLicense:NOASSERTIONStargazers:5Issues:3Issues:0

awesome-crawler

A collection of awesome web crawler,spider in different languages

License:MITStargazers:3Issues:3Issues:0

awesome-web-scraping

List of libraries, tools and APIs for web scraping and data processing.

Language:MakefileLicense:NOASSERTIONStargazers:2Issues:3Issues:0

awesome-flask

A curated list of awesome Flask resources and plugins

awesome-python

A curated list of awesome Python frameworks, libraries, software and resources

Language:PythonLicense:NOASSERTIONStargazers:1Issues:3Issues:0

awesome-python-applications

💿 Free software that works great, and also happens to be open-source Python.

Language:PythonStargazers:1Issues:3Issues:0

public-test

public-test

Language:PythonLicense:GPL-3.0Stargazers:1Issues:3Issues:0

Python-Algorithms

All Algorithms implemented in Python

Language:PythonLicense:MITStargazers:1Issues:3Issues:0

scrapy-redis

Redis-based components for Scrapy.

Language:PythonLicense:MITStargazers:1Issues:3Issues:0
Language:PythonLicense:BSD-3-ClauseStargazers:1Issues:3Issues:0

scrapyd-cluster-on-heroku-scrapydweb-app-git

How to set up Scrapyd cluster on Heroku

Language:PythonLicense:GPL-3.0Stargazers:1Issues:3Issues:0

queuelib

Collection of persistent (disk-based) queues

Language:PythonLicense:BSD-3-ClauseStargazers:0Issues:3Issues:0

scrapyd-cluster-on-heroku-scrapyd-app-basic-auth

scrapyd-cluster-on-heroku-scrapyd-app-basic-auth

Language:PythonLicense:GPL-3.0Stargazers:0Issues:3Issues:0

scrapyd-cluster-on-heroku-scrapyd-app-git

How to set up Scrapyd cluster on Heroku

Language:PythonLicense:GPL-3.0Stargazers:0Issues:2Issues:0

scrapyd-cluster-on-heroku-scrapydweb-app

How to set up Scrapyd cluster on Heroku

Language:PythonLicense:GPL-3.0Stargazers:0Issues:3Issues:0

temp

temp

Language:HTMLStargazers:0Issues:2Issues:0