Read streams do not buffer, resulting in multiple redundant HTTP requests.

[Andrew Scott]

Which service(blob, file, queue, table) does this issue concern?

File, but probably also Blob

Which version of the SDK was used?

Files.Shares 12.8.0

Which platform are you using? (ex: .NET Core 2.1)

.NET 6.0.1

What problem was encountered?

When opening a stream and feeding it into an API that will perform a few different reads of the stream, it appears that a new HTTP request is made for each read of the stream.

I'd expect that only a single GET is performed. This was the case for the Microsoft.WindowsAzure.Storage SDK.

How can we reproduce the problem in the simplest way?

Run the following, putting a URI for an image in the appropriate spot.

Observe the console output. See there's one HEAD request (expected), followed by multiple duplicate GET requests (expected only one).
All GET requests are made with the same x-ms-range header representing the full byte range of the file.

using System.Collections.Concurrent;
using System.Diagnostics;
using Azure.Storage.Files.Shares;

var httpListener = new HttpMonitoringListener();
DiagnosticListener.AllListeners.Subscribe(httpListener);

var file = new ShareFileClient(new Uri("<SAS uri of an image - the one I tested was a 44.8KB PNG file>"));
var stream = await file.OpenReadAsync();

var gdiImage = System.Drawing.Image.FromStream(stream);

public class HttpMonitoringListener : IObserver<DiagnosticListener>, IObserver<KeyValuePair<string, object>>
{
    public void OnCompleted() { }
    public void OnError(Exception error) { }
    public void OnNext(DiagnosticListener listener) { if (listener.Name == "HttpHandlerDiagnosticListener") listener.Subscribe(this); }

    int i = 1;
    public void OnNext(KeyValuePair<string, object> value)
    {
        if (value.Key == "System.Net.Http.HttpRequestOut.Start")
        {
            var request = ((HttpRequestMessage)value.Value.GetType().GetProperty("Request").GetValue(value.Value));

            Console.WriteLine($"======= Request #{i++} =======");
            Console.Write(request.Method + " ");
            Console.WriteLine(request.RequestUri);
            Console.WriteLine(request.Headers);
        }
    }
}

Output:

Details

======= Request #1 =======
GET https://<url removed>
x-ms-version: 2020-10-02
Accept: application/xml
x-ms-client-request-id: 74421d50-d353-4ba8-ad38-f37c41b311f6
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Files.Shares/12.8.0 (.NET 6.0.1; Microsoft Windows 10.0.19043)
traceparent: 00-c3fb475d27f3cc346bdb8fee324df91d-54895f9a961eb8a2-00

======= Request #2 =======
GET https://<url removed>
x-ms-version: 2020-10-02
x-ms-range: bytes=0-4194303
x-ms-range-get-content-md5: false
Accept: application/xml
x-ms-client-request-id: 4fa4ffac-7e76-47df-a637-bafbe715ad15
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Files.Shares/12.8.0 (.NET 6.0.1; Microsoft Windows 10.0.19043)
traceparent: 00-7236457e0a74370761f4278105f84729-e598de71a40b9258-00

======= Request #3 =======
GET https://<url removed>    
x-ms-version: 2020-10-02
x-ms-range: bytes=0-4194303
x-ms-range-get-content-md5: false
Accept: application/xml
x-ms-client-request-id: 393710ce-07e2-469b-ad4d-c491b661ec0f
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Files.Shares/12.8.0 (.NET 6.0.1; Microsoft Windows 10.0.19043)
traceparent: 00-14b312a86fb8ff5fa38d7a90bdb42fde-10814ba33d601a57-00

======= Request #4 =======
GET https://<url removed>   
x-ms-version: 2020-10-02
x-ms-range: bytes=0-4194303
x-ms-range-get-content-md5: false
Accept: application/xml
x-ms-client-request-id: 7ffe8dd8-6bcb-40e5-a20f-f0f59056ad2f
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Files.Shares/12.8.0 (.NET 6.0.1; Microsoft Windows 10.0.19043)
traceparent: 00-072de5ec78830ebfe1e68da00643687e-af5018c0c605ebf8-00

======= Request #5 =======
GET https://<url removed>   
x-ms-version: 2020-10-02
x-ms-range: bytes=0-4194303
x-ms-range-get-content-md5: false
Accept: application/xml
x-ms-client-request-id: c163bda7-1ce0-4df5-b0f5-7ef6d30d7353
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Files.Shares/12.8.0 (.NET 6.0.1; Microsoft Windows 10.0.19043)
traceparent: 00-21e86590502777720398f421d58e7b23-4298db6c9005f0c2-00

======= Request #6 =======
GET https://<url removed>
x-ms-version: 2020-10-02
x-ms-range: bytes=0-4194303
x-ms-range-get-content-md5: false
Accept: application/xml
x-ms-client-request-id: f1f6f888-6821-4c60-8fe4-d557328a1e4e
x-ms-return-client-request-id: true
User-Agent: azsdk-net-Storage.Files.Shares/12.8.0 (.NET 6.0.1; Microsoft Windows 10.0.19043)
traceparent: 00-38caf4775b4590a38fb5cce4eb43ae44-37bcee204c0c0bc3-00

Have you found a mitigation/solution?

Copy the stream from the Azure SDK into a MemoryStream first, then feed it into the target API.

Providing an explicit bufferSize parameter to OpenReadAsync has no effect.

[Andrew Scott]

Oh, I somehow ended up in the wrong repo.