Giters

robinrodricks / FluentFTP

An FTP and FTPS client for .NET & .NET Standard, optimized for speed. Provides extensive FTP commands, File uploads/downloads, SSL/TLS connections, Automatic directory listing parsing, File hashing/checksums, File permissions/CHMOD, FTP proxies, FXP support, UTF-8 support, Async/await support, Powershell support and more. Written entirely in C#.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host

robinrodricks opened this issue · comments

commented

NOTE FROM ADMIN:

If you have this issue, please add your post below instead of creating duplicate issues!

First try this, it disables polling.

client.SocketPollInterval = 0;

And try this, it reduces the polling interval to ensure that the connection does not time-out.

client.SocketPollInterval = 1000;

Thanks.

Commonly occurs in:

  • GetListing()
  • DeleteDirectory()

For all the long running methods like UploadFileInternal and DowloadFileInternal and the above, we need to ensure that we poll the server to ensure its still connected. Could this be a fix to the "forcibly closed" error that we commonly receive?

commented

I think I need to reread the spec, but can the connection close while data is being actively streamed to it though?

commented

@jblacker Good point. I created this "catch all" issue in case people were having trouble. I can't exactly explain or reproduce it. It just keeps coming up time and again so I assumed something was wrong with the methods. Also, I don't really know how/when Polling is done because it was written by JP Trosclair a long time ago.

commented

Yeah the polling is happening on the Socket level deep in the weeds in FtpSocketStream, but determining exactly when it happens will take some serious debugging. I noticed that whenever it does get called it's always like 4 times in quick succession.

commented

Amazing info. The 4 times sounds like a bug - I think once is enough?

commented

Yeah, I'd imagine once per interval would be enough. You can clearly see it happening in the log file posted in #99 (comment)

commented

I'm using FluentFTP for the first time and having a similar issue.

My usage scenario is that every minutes it:
Connect, GetListing (return over 4000 items), Download 3 of these items, Disconnect.
There's no sleep or downtime or anything in between.
It fails more often than it succeed, but maybe once every 3 try it succeed.

The Connect always succeed, but either the GetListing or the Download throw me a System.Net.Sockets.SocketException (0x80004005)

I used FileZilla to test the ftp server and can connect/disconnect and download any files hundreds of times and never encountered any error, so I believe it's a problem in the lib and not the server.

log.txt

commented

I ended up dropping FluentFTP and using System.Net.FtpWebRequest directly, and it works all the time.

commented

catch this error while uploading 1 file (1KB). Is there any workaround ?

commented

I have the same problem.
I detect that the connection was closed every 137 files downloaded !

commented

After differents tests, it would seem there is a timeout on the data connection !
I tried to define DataConnectionTimeout propertie but no change.

commented

I've got this situation, too!!

My Test environment like this:
Framework: .Net Framework 4.0
FluentFTP : 18.02
Remote FTP Server: SunOS
Test Result :
=> [OK] Windows 7 Ent -> SunOS
=> [OK] Windows 10 Home -> SunOS
=> [Failed] Windows Server 2012 -> SunOS

Architecture: x64

Help me~~~

commented

Same issue here downloading huge files. No news?

commented

Just in case there was any doubt this issue is still present.

For me, it is occurring after listing a bunch of directories, and downloading a bunch of files (hundreds) at very high speed.

It does not occur at lower network speeds.

It's actually predictable, and will occur on the same file every time if the remote directory contents remains unchanged.

I've also found that simply retrying the download causes the file contents to be incorrect. To work around, I've had to add Disconnect() and Connect() before retrying.

I am really tempted to switch back to the native .NET FTP support, except it doesn't support extended FTP functionality such as CHMOD.

commented

I'm getting the exception with v33.1.0, v35.0.5 and a few versions in between (have not tested them all)
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

But using v33.0.3 works without issues, exact same code. Release notes don't list any change that could trigger this.

Setting SocketPollInterval does not help. My console app is running on .NET FW 4.7.2.
Connection is FTPS w/ Implicit mode.

Logging info with failing version (redacted):


16:52:58.0518 DEBUG S01 FtpClient: >         Connect()
16:52:58.0648 DEBUG S01 FtpClient: Status:   Connecting to xxx.xxx.xxx.xxx:990
16:52:58.2148 DEBUG S01 FtpClient: >         Dispose()
16:52:58.2148 DEBUG S01 FtpClient: Status:   Disposing FtpClient object...
16:52:58.2148 DEBUG S01 FtpClient: Status:   Disposing FtpSocketStream...
16:52:58.2148 DEBUG S01 FtpClient: Status:   Disposing FtpSocketStream...

Logging info with 33.0.3 (redacted):

16:56:21.7104 DEBUG S01 FtpClient: Status:   Time to activate encryption: 0h 0m 0s.  Total Seconds: 0,1640041.
16:56:21.7304 DEBUG S01 FtpClient: Response: 220 Welcome to the xxxxxxxxxxxxxxxx
16:56:21.7484 DEBUG S01 FtpClient: Command:  USER xxxx
16:56:21.7754 DEBUG S01 FtpClient: Response: 331 User xxxx, password please
16:56:21.7774 DEBUG S01 FtpClient: Command:  PASS ***
16:56:21.7974 DEBUG S01 FtpClient: Response: 230 Password Ok, User logged in
16:56:21.7974 DEBUG S01 FtpClient: Command:  PBSZ 0
16:56:21.8174 DEBUG S01 FtpClient: Response: 200 PBSZ=0
16:56:21.8174 DEBUG S01 FtpClient: Command:  PROT P
16:56:21.8344 DEBUG S01 FtpClient: Response: 200 PROT P OK, data channel will be secured
16:56:21.8344 DEBUG S01 FtpClient: Command:  FEAT
16:56:21.8594 DEBUG S01 FtpClient: Response: 211- Additional features supported include:
Response: MDTM
Response: MFCT
Response: MFMT
Response: SIZE
Response: REST STREAM
Response: AUTH TLS
Response: AUTH SSL
Response: PBSZ
Response: PROT
Response: EPRT
Response: EPSV
Response: XCRC
Response: XSHA1
Response: XSHA256
Response: XSHA512
Response: XMD5
Response: HASH SHA-256;SHA-512;SHA-1*;MD5
Response: LANG EN*
Response: SITE PSWD
Response: SITE ZONE
Response: SITE UTIME
Response: MODE Z ZLIB(LEVEL)
Response: MLST Type*;Size*;Modify*;Create*;
Response: CLNT
Response: CSID
Response: RMDA
Response: UTF8
16:56:21.8724 DEBUG S01 FtpClient: Response: 211 End
16:56:21.8884 DEBUG S01 FtpClient: Status:   Text encoding: System.Text.UTF8Encoding
16:56:21.8884 DEBUG S01 FtpClient: Command:  OPTS UTF8 ON
16:56:21.9084 DEBUG S01 FtpClient: Response: 200 UTF8 support on
16:56:21.9084 DEBUG S01 FtpClient: Command:  SYST
16:56:21.9304 DEBUG S01 FtpClient: Response: 215 UNIX Type: L8

If I need to submit more info, I'm happy to help.

commented

I did a test (like stress test) that connects to the FTPs server and download files.
Here is the source code of the test project

The test is executed in a loop, about 170 iterations.

One iteration is to

  • check folder exists (FTPS fails if GetListing is executed for missing folder)
  • get folder listing
  • download each file

170 iterations, 3 folders, 2 files in each folder.

I've tried the FileZilla ftp server (both 1.1.0-beta1 and 1.0.1) and IIS Ftps server - errors are the same with slightly variations.

IIS ftps

I've used the https://medium.com/@sithum/create-ftps-server-on-windows-server-iis-in-5-minutes-sithum-devops-7a09823d91a4 guide to set up ftps with IIS

The problem is that I'm getting error each 25 time - GetListing returns nothing for 183085 (however there are files)

I can run test project three times in a row and three times it produces the same errors

  • 25th iteration could not get 183085 folder content.
  • Then error An existing connection was forcibly closed by the remote host is thrown for GetListing and 680605 folder on 49th iteration.
  • then DirectoryExists check throws the same connection closed for notexists directory.
  • and then file download fails /183085/Mzc3w_4nTee1.jpg

The same (approximately) errors are thrown each time I'm running this test FluentFtpMT

FileZilla

I've just installed FileZilla server and added a test user

with the FileZilla server

  • each 57th iteration fails to GetListing on dir failed to get dir 680605
  • each 86th iteration fails to download file failed to get file /183085/WdZ6redPXm7j.jpg
  • 114th iteration fails to GetListing of 680605 directory
  • 143th iteration fails to download file failed to get file /183085/WdZ6redPXm7j.jpg

There are less errors on FileZilla with 500ms pauses between each ftp call but

  • iterating 80

    2021-10-28 17:09:16,903 [1] WARN - no folder listing result on path:680605

  • iterating 109

2021-10-28 17:11:54,444 [1] WARN - no folder listing result on path:183085

  • iterating 137

    2021-10-28 17:14:25,551 [1] WARN - no folder listing result on path:220924

here are logs

I did those tests on my Windows 10 OS version 20H2 OS build 19042, 1288)

test project uses the .Net Framework 4.8 and FluentFtp 35.0.5

Connection is FTPS w/ Explicit mode

I've tried to add some Thread.Sleep between ftp calls (it may done using the Websettings.Wait = true) but errors are still here

commented

@oleksabor Using your exact github code and connecting to a very high performace proftpd server of my own, I can reproduce the issue.

I am getting the same errors ( your sln is using FluentFTP V35).

After upgrading to FluentFTP V42 and making the few neccessary ->V40 migration changes, the problem STILL can be reproduced.

So now we can start to search for the problem.

commented

Sorry I enlisted to the army and can't assist you or test any fix if available.

commented

Some first results of testing, using @oleksabor initial approach as a basis.

The goal was to find out a little bit more about the real problem and to remove conjecture.

  1. I have modified the test program to only do the following (up to 500 times):
CWD /home/testuser
PWD

These commands do not use a data connection. No data connection (SSL) needs to be created and disposed. They only exercise the control connection.

These commands are iterated up to 500 times with no waits in between.

They always fail in the same way on the 474th iteration, on all of my available client systems.

  1. The test suite will run all 500 iterations without failure if encryption is turned off.

  2. The test suite encounters the problem or multiple problems if encryption is turned on.

  3. Disabling stream buffering does not help.

  4. Disabling or enabling socket polling does not have any effect on the problem occurrence(s).

  5. .NET 6.0 or .NET Framework 4.8 targets both exhibit the same problem(s).

  6. Any problems occur after the same amount of iterations - deterministic and not random. Independent of host system.

  7. The errors occur in GetReply...ReadLine --> FtpSocketStream.ReadLine --> Read. I put a try/catch into FtpSocketStream.Read: The errors themselves are always, (note: also in the original Getlisting/Download szenario) "an attempt was made to access a socket in a way forbidden by its access permissions", an IOException, after which the socket is no longer connected, which is reacted to in different ways when they percolate upwards, depending on the function that you called. In a GetListing(), in a Download..., depending on the code that catches this exception, the further behaviour difffers. The control connection is closed though, which can cause a reconnect attempt, thus causing confusion if you don`t see that happening, or you can get "empty directories". Standardizing this behaviour could be a todo-item.

  8. It looks like continuous reading from the SSL control connection stream in a huge amount of single byte READ operations is somehow exhausting some resource inside the implementation of SslStream.

commented

So it seems that in this thread we are talking about two sepearate issues:

  1. Long running processes, like a huge GetListing or a huge Download/Upload, getting terminated by the server suddenly shutting down the control connection.

  2. Strange errors after performing large amounts of commands on the control connection

Long running processes

That is the original post and its title corresponds to the error you would see:
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host](https://github.com/robinrodricks/FluentFTP/issues/104#top)

The following was mentioned in this issue, related to this issue:

a. Polling
b. NOOP commands

Polling is done whenever a FluentFTP method causes IsConnected to be queried (and of course, the last socket activity is longer ago than the polling interval). Polling ensures that our socket is kept "up-to-date" concerning its status, thus avoiding exceptions when trying to write to a closed socket. It does not cause a socket "to be kept alive if idle" and does not help for long running transfers on the data connection while the control connection is idle for a long time.

NOOP commands are actually the accepted way of keeping the control connection alive for long running transfers on the data connection.

This issue is actually fixed.

Strange errors

After performing large amounts of Read operations on the SslStream socket, an IOException occurs with message an attempt was made to access a socket in a way forbidden by its access permissions

This is currently being investigated. See previous post.

I will open a separate issue for this problem. This issue here can be closed, the solution is to use NOOPs, see the Wiki for this.