Exceen / 4chan-downloader

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

"Something went wrong"

necrorika opened this issue · comments

commented

Regardless of thread, board, script location, drive letter, attribute, or host machine the script returns "Something went wrong" and times out after several tries.
image
Above picture showing different threads and boards and different attribute flags, all three failing almost immediately.
image
Interestingly however it works fine on an arch install. I've tried it on two machines running Windows 10, reinstalled python 3 as well
Not entirely sure why it's dropping almost immediately on Windows. Assume this is a Windows problem, but nothing changed since I used it yesterday. will close if I find a solution to using it on windows.
sorry there isn't much information to go off of.

This was thrown by this block of code:

except (urllib.error.URLError, http.client.BadStatusLine, http.client.IncompleteRead):
    if not args.less:
        log.warning('Something went wrong')

In hindsight, it was silly not to include a statement right there to debug what the thing that "went wrong", so I'll include it in my own fork and you maybe you can post your results here?

commented

This was thrown by this block of code:

except (urllib.error.URLError, http.client.BadStatusLine, http.client.IncompleteRead):
    if not args.less:
        log.warning('Something went wrong')

In hindsight, it was silly not to include a statement right there to debug what the thing that "went wrong", so I'll include it in my own fork and you maybe you can post your results here?

Here ya go:
image

It's not ideal, but try using http://boards.4chan.org/ instead of https. Maybe that'll work.

commented

Nope, same thing happens unfortunately
image

Following some StackOverflow answers, I think you should run

pip install --upgrade certifi

This will upgrade your SSL certificates. If they're expired/corrupted, it can cause issues with urllib and Requests.

commented

Still returns the same errors.
I upgraded the certificates and still the same thing, so I went and updated pip as well, still nothing. Went and added the trusted hosts to the pip.ini file, still the same thing as well
so, overall, same thing.

Can you post the output of your

pip --version

and

python --version
commented

image
looks like the most up to date python version is 3.9.7 so I'll probably update that in the morning if I have time and report back if all went well

commented

Little late, oops
Now returns this error after updating to Python 3.10:
image

pip and python versions:
image

There must be a way to run the program without verifying the SSL certificates.

@cherioux
Could you please checkout this branch and tell me if it works? You need to run the script with the added -i/--ignore-ssl-verification-flag

https://github.com/Exceen/4chan-downloader/tree/ssl-verification

File ".\inb4404.py", line 43
if args.ignore_ssl_verification:
^
IndentationError: unindent does not match any outer indentation level

So I fixed the indent level, which was easy.

Got this.

Traceback (most recent call last):
File ".\inb4404.py", line 192, in
main()
File ".\inb4404.py", line 32, in main
download_thread(thread, args)
File ".\inb4404.py", line 92, in download_thread
html_result = load(thread_link).decode('utf-8')
File ".\inb4404.py", line 51, in load
ctx = ssl.create_default_context()
NameError: name 'ssl' is not defined

Instead of import ssl in main(), I just declared at the top, and I don't have an issue any more.
Is there anything wrong with just import ssl always? I'm a dev, just not a python dev. Not sure on semantics or anything.

I'm really sorry, must have somehow uploaded something wrong. I fixed these import-issues now. Although it still needs testing if this flag fixes the SSL certificate issues.

Oh, guess I forgot to include that with this flag, it does indeed work!

commented

I'm late, but this does work!