vimagick / dockerfiles

:whale: A curated list of delicious docker recipes 🇺🇦🇮🇱 (Let's Fight Against ************)

Home Page:https://hub.docker.com/u/vimagick/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can't schedule spiders to work - scrapyd

opened this issue · comments

{"node_name": "4392081d8bcd", "status": "error", "message": "/usr/local/lib/python3.7/dist-packages/scrapy/utils/project.py:94: ScrapyDeprecationWarning: Use of environment variables prefixed with SCRAPY_ to override settings is deprecated. The following environment variables are currently defined: VERSION\n ScrapyDeprecationWarning\n"}

when I run 'curl http://localhost:6800/schedule.json -d project=myproject -d spider=somespider', the above is the output I get, is there a way to get around this?

@ghost Did you manage to handle it ? Having the same issue...

I've moved scrapyd to https://github.com/EasyPi/docker-scrapyd
I'm running it without any problem.

version: "3.8"

services:

  scrapyd:
    image: easypi/scrapyd
    ports:
      - "6800:6800"
    volumes:
      - ./data:/var/lib/scrapyd
      - /usr/local/lib/python3.9/dist-packages
    restart: unless-stopped