"Connection reset" error when file downloaded

Question

"Connection reset" error when file downloaded

Ponywka opened this issue 3 years ago · comments

OS: Windows 11 21H2 (22000.194)
Java version: OpenJDK 11.0.2 / OpenJDK 17.0.1 / Oracle JDK 17
CLI command: java -jar bandcamp-collection-downloader.jar -j 1 -r 100 -f mp3-320 -d bandcamp320 ponywka
Album in queue: https://mirroredreality.bandcamp.com/track/pony-girl-mirroredreality-remix

When the file has been downloaded completely, the program gives an error and tries to download the file again.

Ezwen · Answer 1 · Tue Nov 30 2021 19:56:52 GMT+0800 (China Standard Time)

Interesting, so what you are saying is that even if the downloaded file is complete (as shown by the MD5 hashes), it keeps retrying to download it again?

Strange. Can you easily reproduce the bug?

Vladimir Avtsenov · Answer 2 · Wed Dec 01 2021 01:02:20 GMT+0800 (China Standard Time)

Interesting, so what you are saying is that even if the downloaded file is complete (as shown by the MD5 hashes), it keeps retrying to download it again?

Yep

Can you easily reproduce the bug?

https://20101010.xyz/public/bandcamp-collection-downloader-2021-11-30-19-58-12.mp4

Ezwen · Answer 3 · Tue Dec 07 2021 01:55:44 GMT+0800 (China Standard Time)

I bought the track for 50 cents just to test, and everything went fine. So it's not specific to this item, and is something related to your setup, I suppose. I wonder what this could be.

Vladimir Avtsenov · Answer 4 · Tue Dec 07 2021 05:39:32 GMT+0800 (China Standard Time)

Interesting... 🤔

Vladimir Avtsenov · Answer 5 · Wed Dec 08 2021 03:55:50 GMT+0800 (China Standard Time)

OS: Arch Linux
Java version:

openjdk version "17" 2021-09-14
OpenJDK Runtime Environment (build 17+35)
OpenJDK 64-Bit Server VM (build 17+35, mixed mode)

Same problem
https://asciinema.org/a/Cne8yYoxpQc8LhPqC45CmiNQI

Ezwen · Answer 6 · Wed Dec 08 2021 16:39:13 GMT+0800 (China Standard Time)

I've added an option --debug to display the complete error stack trace, and a test build with this new feature can be downloaded here: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521019/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false

Can you try again with this test build, and report back here the complete error stack trace?

Vladimir Avtsenov · Answer 7 · Wed Dec 08 2021 19:51:50 GMT+0800 (China Standard Time)

Error while trying: "java.net.SocketException: Connection reset".

java.net.SocketException: Connection reset
        at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
        at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
        at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
        at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
        at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
        at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
        at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
        at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
        at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
        at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
        at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
        at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
        at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
        at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
        at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:44)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:303)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:17)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:248)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:245)
        at bandcampcollectiondownloader.util.Util.retry(Util.kt:66)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
        at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
        at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
        at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Vladimir Avtsenov · Answer 8 · Wed Dec 08 2021 20:09:21 GMT+0800 (China Standard Time)

Also in Arch Linux:
https://20101010.xyz/public/bcd-2021-12-08-14-59-33.mp4
00:00 - System info
00:32 - Start downloading & Crashes when downloading singles
01:40 - Downloading albums without chashes

Error while trying: "java.net.SocketException: Connection reset".                                                                 
java.net.SocketException: Connection reset
	at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
	at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
	at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
	at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
	at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
	at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
	at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:44)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:303)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:17)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:248)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:245)
	at bandcampcollectiondownloader.util.Util.retry(Util.kt:66)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Could not download item: Could not perform task after 0 retries.                                                                  
bandcampcollectiondownloader.core.BandCampDownloaderError: Could not perform task after 0 retries.
	at bandcampcollectiondownloader.util.Util.retry(Util.kt:75)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:245)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:119)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:132)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:94)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:14)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Ezwen · Answer 9 · Wed Dec 08 2021 21:14:20 GMT+0800 (China Standard Time)

Great thanks. So the problem strangely arises when trying to download the cover image for the track.

I've added yet more debug printing, to have a better look at what is the URL of the cover file it tries do download. Can you try again with this new test build here, using --debug once more: https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521424/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false

Vladimir Avtsenov · Answer 10 · Wed Dec 08 2021 22:48:33 GMT+0800 (China Standard Time)

As I understand it, in Russia (at least at the Internet provider Rostelecom and Tele2), the domain of pictures at Bandcamp is blocked. This makes sense because usually the Bandcamp page looks like this for me:

Managing item 1/93                                                                                                                
Found release "Pony Girl (MirroredReality Remix)" (2018) by MirroredReality (Bandcamp ID: r152688695).                            
Starting the download of "Pony Girl (MirroredReality Remix)" (2018) by MirroredReality.                                           
[DEBUG] Starting download of: https://p4.bcbits.com/download/track/12c24612d74aee5a3ae64fa9a7c39976d/mp3-320/3850187924?id=3850187924&sig=2f115bb4dd67221746821bd767fd2ea4&sitem_id=152688695&token=1639579443_aa3ad6ca4549c315d25ad2df66e7ac3f55b4fcd4
Progress: 0.00 %                                                                                                                  
...                                                                                              
Progress: 99.99 %                                                                                                                 
                                                                                                                                  
[DEBUG] This is a single track release: fetching the cover                                                                        
[DEBUG] Starting download of: https://f4.bcbits.com/img/a637921479_10                                                             
Error while trying: "java.net.SocketException: Connection reset".                                                                 
[DEBUG] [DEBUG]java.net.SocketException: Connection reset
	at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:323)
	at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350)
	at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803)
	at java.base/java.net.Socket$SocketInputStream.read(Socket.java:966)
	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421)
	at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589)
	at java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308)
	at bandcampcollectiondownloader.util.RealIO.downloadFile(RealIO.kt:43)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadRelease(BandcampCollectionDownloader.kt:315)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.access$downloadRelease(BandcampCollectionDownloader.kt:18)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:258)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader$manageDownloadPage$1.invoke(BandcampCollectionDownloader.kt:255)
	at bandcampcollectiondownloader.util.Util.retry(Util.kt:35)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:255)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:131)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:142)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:96)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:13)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Could not download item: Could not perform task after 0 retries.                                                                  
[DEBUG] [DEBUG]bandcampcollectiondownloader.core.BandCampDownloaderError: Could not perform task after 0 retries.
	at bandcampcollectiondownloader.util.Util.retry(Util.kt:42)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.manageDownloadPage(BandcampCollectionDownloader.kt:255)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll$lambda-1(BandcampCollectionDownloader.kt:131)
	at bandcampcollectiondownloader.core.BandcampCollectionDownloader.downloadAll(BandcampCollectionDownloader.kt:142)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:96)
	at bandcampcollectiondownloader.cli.Command.call(Command.kt:13)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at bandcampcollectiondownloader.main.MainKt.main(Main.kt:15)

Ezwen · Answer 11 · Wed Dec 08 2021 22:57:22 GMT+0800 (China Standard Time)

Ah I see, I did not know that! Well at least now we have nailed the problem :)

Since it will be difficult to detect programmatically whether the problem is a (temporary) real network error, or a (permanent) proper blocking from the ISP, instead we can consider adding a new flag in the app to choose to not download the covers for single-track releases. WDYT?

Vladimir Avtsenov · Answer 12 · Wed Dec 08 2021 23:11:35 GMT+0800 (China Standard Time)

Good idea, but I have this problem already 1 year (no pictures on Bandcamp).
And yep, it is blocked: https://isitblockedinrussia.com/?host=f4.bcbits.com

Ezwen · Answer 13 · Wed Dec 08 2021 23:26:56 GMT+0800 (China Standard Time)

Here is a new test build with a new option --no-covers-single-track : https://framagit.org/Ezwen/bandcamp-collection-downloader/-/jobs/1521643/artifacts/raw/build/libs/bandcamp-collection-downloader.jar?inline=false

Can you try with this option to check if it fixes your problem? I'm not 100% sure I will keep it exactly this way for the next release, but at least we will be sure that this is the right track.

Vladimir Avtsenov · Answer 14 · Wed Dec 08 2021 23:33:18 GMT+0800 (China Standard Time)

Yay! Now it works like I want. Thank you!

Ezwen · Answer 15 · Wed Dec 08 2021 23:36:59 GMT+0800 (China Standard Time)

Fantastic, thanks for your patience and helping finding out the problem! Now the tool should be a little more russia-proof 😁 . I'll probably keep the option like this for the next release, but just in case I decide to rename it, please do check the release notes when/if you will download the next stable version.