No wallpapers downloaded
jerry1333 opened this issue · comments
root@serwerek:/home/infoserwis/wallhaven# ./wallhaven.sh
Download Page 1 - done!
Download Wallpapers from Page 1cat: downloaded.txt: Nie ma takiego pliku ani katalogu
- done!
Download Page 2 - done!
Download Wallpapers from Page 2 - done!
root@serwerek:/home/infoserwis/wallhaven# ls ./walls
downloaded.txt
root@serwerek:/home/infoserwis/wallhaven#Running Debian 6 at root account, parallel download throws more errors.
Could you provide the "Configuration Settings" block? Also please provide the contents of the "downloaded.txt" file.
The first message ist normal (but i guess i should fix it, it only appears the first time the script is run because the downloaded.txt wasnt created yet)
Settings:
#####################################
### Needed for NSFW/Favorites ###
#####################################
# Enter your Username
USER="jerry1333"
# Enter your password
PASS="######"
#####################################
### End needed for NSFW/Favorites ###
#####################################
####################################
### Configuration Options ###
####################################
# Where should the Wallpapers be stored?
LOCATION=/home/infoserwis/wallhaven/walls
# How many Wallpapers should be downloaded, should be multiples of 24 (right now they only use a fixed number of thumbs per pa$
WPNUMBER=48
# Type standard (newest, oldest, random, hits, mostfav), search, favorites (for now only the default collection), useruploads $
TYPE=standard
# From which Categories should Wallpapers be downloaded, first number is for General, second for Anime, third for People, 1 to$
CATEGORIES=111
# Which Purity Wallpapers should be downloaded, first number is for sfw content, second for sketchy content, third for nsfw, 1$
PURITY=110
# Which Resolution should be downloaded, leave empty for all (most common resolutions possible, for details see wallhaven site)
RESOLUTION=
# Which aspectratio should be downloaded, leave empty for all (possible values: 4x3, 5x4, 16x9, 16x10, 32x9, 48x9)
RATIO=
# Which Type should be displayed (relevance, random, date_added, views, favorites)
SORTING=random
# How should the Wallpapers be ordered (desc, asc)
ORDER=desc
# Searchterm, only used if TYPE = search
QUERY="nature"
# User from which Wallpapers should be downloaded (only used for TYPE=useruploads)
USR=AksumkA
# use gnu parallel to speed up the download (0, 1), if set to 1 make sure you have gnuparallel installed, see normal.vs.parall$
PARALLEL=0directory is chmoded 777
downloaded.txt
21076
50767
70983
47459
83965
5333
80913
2012
22202
37157
73551
13510
40790
79103
3512
32208
52686
53885
27194
67593
20520
20495
77690
37660
72220
86978
87511
23484
28348
41701
56314
62142
3337
42178
74497
21084
11550
80833
11321
31311
38758
65760
19542
44881
2404
16182
7560
8648
mmh it works for me, please remove the "-q" from "wget -q ..." in line 112 and 134 and post the output of the script
edit: oh forgot to mention also in line 135
I have wget commands in line: 110, 132, 133.
Output: http://pastebin.com/tF9eGU5v
oh, sorry i forgot that i just made a few changes.
Try replacing line 133 with this:
cat $number | egrep -o 'wallpapers.*(png|jpg|gif)' | echo "http://$(cat -)" | wget --keep-session-cookies --load-cookies=cookies.txt --referer=http://alpha.wallhaven.cc/wallpaper/$number -i -
For some reason your wget version seems to not add the "http://" in front of the url, could you also provide your wget version?
wget --version
root@serwerek:/home/infoserwis/wallhaven# wget --version
GNU Wget 1.13.4 zbudowany na systemie linux-gnu.
+digest +https +ipv6 +iri +large-file +nls -ntlm +opie +ssl/gnutls
Wgetrc:
/etc/wgetrc (system)
Lokalizacja: /usr/share/locale
Kompilacja: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc"
-DLOCALEDIR="/usr/share/locale" -I. -I../lib -I../lib
-D_FORTIFY_SOURCE=2 -Iyes/include -g -O2 -fstack-protector
--param=ssp-buffer-size=4 -Wformat -Werror=format-security
-DNO_SSLv2 -D_FILE_OFFSET_BITS=64 -g -Wall
Odnośnik: gcc -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat
-Werror=format-security -DNO_SSLv2 -D_FILE_OFFSET_BITS=64 -g -Wall
-Wl,-z,relro -Lyes/lib -lgnutls -lgcrypt -lgpg-error -lz -lidn -lrt
ftp-opie.o gnutls.o ../lib/libgnu.a
Copyright (C) 2009 Free Software Foundation, Inc.
Licencja GPLv3+: GNU GPL w wersji 3 lub późniejszej
<http://www.gnu.org/licenses/gpl.html>.
Niniejszy program jest oprogramowaniem wolnodostępnym: można go
modyfikować i rozpowszechniać.
Nie ma ŻADNEJ GWARANCJI w zakresie dopuszczalnym przez prawo.
Autor oryginału Hrvoje Niksic <hniksic@arsdigita.com>.
Prosimy o zgłaszanie błędów i propozycji na adres <bug-wget@gnu.org>.
root@serwerek:/home/infoserwis/wallhaven# Does it work with the new Line 133?
After change it's downloading something so it should be fine.
@edit:
Yep it's downloading. Thanks!
Btw. for debian there is no gnuparallel only parallel with syntax:
root@serwerek:/home/infoserwis/wallhaven# parallel --help
Usage:
parallel [options] [command [arguments]] < list_of_arguments
parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))...
cat ... | parallel --pipe [options] [command [arguments]]
-j n Run n jobs in parallel
-k Keep same order
-X Multiple arguments with context replace
--colsep regexp Split input on regexp for positional replacements
{} {.} {/} {/.} {#} Replacement strings
{3} {3.} {3/} {3/.} Positional replacement strings
-S sshlogin Example: foo@server.example.com
--slf .. Use ~/.parallel/sshloginfile as the list of sshlogins
--trc {}.bar Shorthand for --transfer --return {}.bar --cleanup
--onall Run the given command with argument on all sshlogins
--nonall Run the given command with no arguments on all sshlogins
--pipe Split stdin (standard input) to multiple jobs.
--recend str Record end separator for --pipe.
--recstart str Record start separator for --pipe.
See 'man parallel' for details
When using GNU Parallel for a publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
It is called with parallel ... but the full name is "GNU Parallel", part of your parallel -help output: ... When using GNU Parallel for a publication please cite: ... see line ~140 cat download.txt | parallel --no-notice wget...
PARALLEL=1
root@serwerek:/home/infoserwis/wallhaven# ./wallhaven.sh
Download Page 1 - done!
Download Wallpapers from Page 1parallel: Error: Command (--no-notice) starts with '-'. Is this a wrong option?
parallel: Error: Command (--no-notice) starts with '-'. Is this a wrong option?
rm: nie można usunąć `79469': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `58143': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `86319': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `42175': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `46468': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `69864': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `84349': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `32804': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `82669': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `3379': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `23143': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `74889': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `84519': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `5584': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `31089': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `6363': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `66152': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `12776': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `48076': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `47144': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `73163': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `25863': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `30527': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `28066': Nie ma takiego pliku ani katalogu
- done!
Download Page 2 - done!
Download Wallpapers from Page 2parallel: Error: Command (--no-notice) starts with '-'. Is this a wrong option?
parallel: Error: Command (--no-notice) starts with '-'. Is this a wrong option?
rm: nie można usunąć `35521': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `46374': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `86897': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `50413': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `46062': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `72739': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `11172': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `25473': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `59078': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `5117': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `26899': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `53857': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `54445': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `34716': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `42003': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `72185': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `18218': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `32062': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `42402': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `71483': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `32211': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `52865': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `48381': Nie ma takiego pliku ani katalogu
rm: nie można usunąć `44400': Nie ma takiego pliku ani katalogu
- done!
please provide:
parallel --version
and take a look at
man parallel
and search for "--no-notice" with
/--no-notice
There is no option called --no-notice so I just removed it from file, it's working.
root@serwerek:/home/infoserwis/wallhaven# parallel --version
WARNING: YOU ARE USING --tollef. USE --gnu FOR GNU PARALLEL
GNU parallel 20120422
Copyright (C) 2007,2008,2009,2010,2011,2012 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.
Web site: http://www.gnu.org/software/parallel
When using GNU Parallel for a publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
mmh it seems some Distributions activate(d) --tollef by default, this option was retired in February this Year with Version 20140222, so it shouldnt be present on most Distributions, could you try replacing
parallel --no-notice
with
parallel --gnu --no-notice
if it works for you i would change it in the script.
If the problems are due to your rather old Version of parallel i would not change it, given the fact that most people will have a newer Version of parallel installed.
No problem, as I said it's working when I removed option --no-notice.
yeah, just want to know if this would also fix it, so that other users wont run into the same problems as you (if i remove the --no-notice option the citation comment shows up every time parallel is used)
Nope it's not showing anything.
root@serwerek:/home/infoserwis/wallhaven# ./wallhaven.sh
Download Page 1 - done!
Download Wallpapers from Page 1cat: downloaded.txt: Nie ma takiego pliku ani katalogu
- done!
Download Page 2 - done!
Download Wallpapers from Page 2 - done!
Download Page 3 - done!
Download Wallpapers from Page 3 - done!
Download Page 4 - done!
Download Wallpapers from Page 4 - done!
Download Page 5 - done!
Download Wallpapers from Page 5 - done!
Download Page 6 - done!
Download Wallpapers from Page 6 - done!
Download Page 7 - done!
Download Wallpapers from Page 7 - done!
Download Page 8 - done!
Download Wallpapers from Page 8 - done!
Download Page 9 - done!
Download Wallpapers from Page 9 - done!
Download Page 10 - done!
Download Wallpapers from Page 10 - done!
Download Page 11 - done!
Download Wallpapers from Page 11 - done!
Download Page 12 - done!
Download Wallpapers from Page 12 - done!
Download Page 13 - done!
Download Wallpapers from Page 13 - done!
Download Page 14 - done!
Download Wallpapers from Page 14File already downloaded!
- done!
Download Page 15 - done!
Download Wallpapers from Page 15 - done!
Download Page 16 - done!
Download Wallpapers from Page 16 - done!
Download Page 17 - done!
Download Wallpapers from Page 17 - done!
Download Page 18 - done!
Download Wallpapers from Page 18 - done!
Download Page 19 - done!
Download Wallpapers from Page 19 - done!
Download Page 20 - done!
Download Wallpapers from Page 20 - done!
Download Page 21 - done!
Download Wallpapers from Page 21 - done!
Download Page 22 - done!
Download Wallpapers from Page 22 - done!
Download Page 23 - done!
Download Wallpapers from Page 23 - done!
Download Page 24 - done!
Download Wallpapers from Page 24 - done!
Download Page 25 - done!
Download Wallpapers from Page 25 - done!
Download Page 26 - done!
Download Wallpapers from Page 26 - done!
Download Page 27 - done!
Download Wallpapers from Page 27 - done!
Download Page 28 - done!
Download Wallpapers from Page 28 - done!
Download Page 29 - done!
Download Wallpapers from Page 29 - done!
Download Page 30 - done!
Download Wallpapers from Page 30 - done!
Download Page 31 - done!
Download Wallpapers from Page 31 - done!
Download Page 32 - done!
Download Wallpapers from Page 32 - done!
Download Page 33 - done!
Download Wallpapers from Page 33 - done!
Download Page 34 - done!
Download Wallpapers from Page 34 - done!
Download Page 35 - done!
Download Wallpapers from Page 35File already downloaded!
- done!
Download Page 36 - done!
Download Wallpapers from Page 36File already downloaded!
- done!
Download Page 37 - done!
Download Wallpapers from Page 37File already downloaded!
- done!
Download Page 38 - done!
Download Wallpapers from Page 38 - done!
Download Page 39 - done!
Download Wallpapers from Page 39File already downloaded!
- done!
Download Page 40 - done!
Download Wallpapers from Page 40 - done!
mmh for me it does
Download Wallpapers from Page 1When using programs that use GNU Parallel to process data for publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
This helps funding further development; and it won't cost you a cent.
To silence this citation notice run 'parallel --bibtex' once or use '--no-notice'.
maybe this comment was added in a later Version, i think i will just leave it as it is and add a comment to the Readme regarding older parallel Versions
Sure. Have a nice day / night. Thanks for fixing script.