crawl: error: Invalid -s value, use -s NAME=VALUE
saavedra29 opened this issue · comments
When scrapyd runs a spider with scrapy==2.8.0 no log files are created inside the logs folder (logs_dir). Using scrapy==2.7.1 works perfectly.
+1
Please upgrade to Scrapyd 1.4.0.
Using scrapy==2.8.0 and scrapyd==1.4.0 still got error , and logs not created in the logs_dir.
scrapyd-01 | 2023-02-10T12:07:39+0800 [-] Process started: project='my-spiders' spider='test-spider' job='74738e4ea8f811ed9b1502420a110703' pid=13 log='logs/my-spiders/test-spider/74738e4ea8f811ed9b1502420a110703.log' items='file:///scrapyd/items/my-spiders/test-spider/74738e4ea8f811ed9b1502420a110703.jl'
scrapyd-01 | 2023-02-10T12:07:40+0800 [Launcher,13/stderr] Usage
scrapyd-01 | =====
scrapyd-01 | scrapy crawl [options] <spider>
scrapyd-01 | crawl: error: Invalid -s value, use -s NAME=VALUE
@jpmckinney Please have a look, consider reopen this issue?
crawl: error: Invalid -s value, use -s NAME=VALUE
That message is unusual. Can you edit your copy of scrapy/commands/__init__.py, and change the line raise UsageError("Invalid -s value, use -s NAME=VALUE", print_help=False), to raise UsageError(repr(opts.set)) ?
@saavedra29 Please provide your scrapyd log (like in #473 (comment)). Otherwise, this is impossible to debug. It was tested and it works in normal conditions.
Edit: Nevermind - if downgrading to Scrapy 2.7.1 fixed your issue, then you must not be using Scrapyd 1.4.0 yet.
@imaxwen Alternatively, install this testing branch:
pip install -e git+https://github.com/scrapy/scrapyd.git@launcher-log#egg=scrapyd
Aha, it is because the FEEDS argument wasn't properly encoded. 1.4.1 is released.
thanks Very efficient 😆