TeamHG-Memex / autologin-middleware

Scrapy middleware for the autologin

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Describe usage with scrapy-splash in more detail

lopuhin opened this issue · comments

A scrapy-splash autologin spider that uses any non-custom Splash endpoint will not work correctly, because cookies argument is not supported, and even though the initial request will use cookies (set via the header argument), subsequent requests on the same page (made by JS for example) will not use cookies and will not be authenticated.
Currently the only solution is to use a custom splash script. We should at least have an example scrapy project that correctly uses all the components.

Done in #16 - added docs and a splash helper