Scrapy Plugins's repositories
scrapy-splash
Scrapy+Splash for JavaScript integration
scrapy-playwright
🎭 Playwright integration for Scrapy
scrapy-djangoitem
Scrapy extension to write scraped items using Django models
scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
scrapy-jsonrpc
Scrapy extension to control spiders using JSON-RPC
scrapy-deltafetch
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
scrapy-magicfields
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
scrapy-jsonschema
Scrapy schema validation pipeline and Item builder using JSON Schema
scrapy-monkeylearn
A Scrapy pipeline to categorize items using MonkeyLearn
scrapy-zyte-api
Zyte API integration for Scrapy
scrapy-pagestorage
A scrapy extension to store requests and responses information in storage service
scrapy-querycleaner
Scrapy spider middleware to clean up query parameters in request URLs
scrapy-splitvariants
Scrapy spider middleware to split an item into multiple items using a multi-valued key
scrapy-dotpersistence
A scrapy extension to sync `.scrapy` folder to an S3 bucket
scrapy-crawlera-fetch
Scrapy Downloader Middleware for Crawlera Fetch API
scrapy-bigml
Scrapy pipeline for writing items to BigML datasets
scrapy-hcf
Scrapy spider middleware to use Scrapinghub's Hub Crawl Frontier as a backend for URLs
scrapy-snowflake-stage-exporter
Snowflake database loading utility with Scrapy integration
scrapy-feedexporter-dropbox
Scrapy feed exporter for Dropbox
scrapy-feedexporter-onedrive
Export to OneDrive
scrapy-settings-log
An extension that allows a user to display all or some of their scrapy spider settings at runtime.