[BUG]: OOM Downloading large Workflow Artifacts

[rickardgranberg]

What happened?

With the release of v4 of https://github.com/actions/upload-artifact, backend changes has been implemented which seems to cause issues when trying to download artifacts.

When trying to download a large (>1GiB) artifact using the following code:

var sm = await client.Actions.Artifacts.DownloadArtifact(owner, reponame, id, "zip");
using (var fs = File.OpenWrite("C:\\temp\\artifact.zip"))
{
 await sm.CopyToAsync(fs);
}

We get OOM:

System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Received an unexpected EOF or 0 bytes from the transport stream.
   at System.Net.ConnectStream.EndRead(IAsyncResult asyncResult)
   at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.EndRead(IAsyncResult asyncResult)
   at System.Net.Http.StreamToStreamCopy.BufferReadCallback(IAsyncResult ar)
   --- End of inner exception stack trace ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Octokit.Internal.HttpClientAdapter.<BuildResponse>d__5.MoveNext() in /_/Octokit/Http/HttpClientAdapter.cs:line 98
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Octokit.Internal.HttpClientAdapter.<Send>d__3.MoveNext() in /_/Octokit/Http/HttpClientAdapter.cs:line 52
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Octokit.Connection.<RunRequest>d__68.MoveNext() in /_/Octokit/Http/Connection.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Octokit.Connection.<GetRawStream>d__65.MoveNext() in /_/Octokit/Http/Connection.cs:line 724
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Octokit.ApiConnection.<GetRawStream>d__13.MoveNext() in /_/Octokit/Http/ApiConnection.cs:line 148
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)

A smaller (~200 MiB) artifacts seems to work fine. As a sidenote, we run this in a 32-bit process, so the OOM happens sooner there I would assume.

A noticeable change in behaviour is that before v4, the DownloadArtifact(...) call would return reasonably quickly, and the processing would then be handled by the CopyToAsync. After v4, DownloadArtifact(...) only returns once all of the data has been dowloaded (in the case of smaller artifacts) or OOMs.

Versions

Octokit 9.0.0

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[rickardgranberg]

I can confirm this is caused by the problem reported in #2898 and fixed by #2905

[Keegan Campbell]

👍 In that case, I'll close this issue. Feel free to reopen if it occurs later.