[Blazor] [WASM] Large File Upload Via Stream not Work as Expected + Memory Leak

[dmanchura]

Describe the bug

Using InputFile I'm trying to drag/drop a 2Gb file and upload as binary to a cloud directory. Using HttpClient + StreamContent I would expect the data to stream from the file to the end-point without much memory increasing. Instead I see two things happen:
File streams into memory (from task manager the memory of a different Edge tab increases rapidly and disk utilization is high).
Out of memory exception is raise or if using a file < 2GB then the file begins to stream to the end-point (Task manager shows network utilization at what you would expect for data upload).

We use the same structure in a WPF application which stream file -> server and I have confirmed it doesn't buffer the whole file to memory first. It reads X length then writes X length.
Looking at the internals of the StreamContent the buffer size is set to the file's length and I do not see how to change that behavior.

It also appears that the memory is not cleared after the request is complete. Looking at task manager and taking a heap snapshot there will be 2GB of JSArrayBufferData objects. These do not clear and trying to upload a second or third file just adds to the memory until the max is reached ~2GB then no more files can upload. (The image is of a ~667Mb file).

Disposing of the Stream, HttpClient, StreamContent doesn't appear to help.

To Reproduce

Have a FileInput. Try upload a large file using StreamContent

Exceptions (if any)

Out of memory exception if the file is larger than 2Gb or even if the file is 600Mb.

Code

<InputFile OnChange="@LoadFiles" multiple/>  
@code{

async Task LoadFiles(InputFileChangeEventArgs evs)
{
    foreach(var file in evs.GetMultipleFiles())
    {
        try
        {
            using var str = file.OpenReadStream(maxAllowedSize: 2L * 1024 * 1024 * 1024);
            using var content = new StreamContent(str);
            var name = "TestBlazorMemory";

            using (var client = new HttpClient{ BaseAddress = new Uri(_uploadUrl)})
            {
                await client.PutAsync("", content);
            }
        }
        catch(Exception e)
        {
            //OOM here. Or even if not OOM memory remains until page changes
        }
    }
}

Further technical details

• .Net SDK 5.0.201
  .NET SDK (reflecting any global.json):
  Version: 5.0.201
  Commit: a09bd5c86c

Runtime Environment:
OS Name: Windows
OS Version: 10.0.18363
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\5.0.201\

Host (useful for support):
Version: 5.0.4
Commit: f27d337295

.NET SDKs installed:
1.1.14 [C:\Program Files\dotnet\sdk]
2.1.522 [C:\Program Files\dotnet\sdk]
2.2.402 [C:\Program Files\dotnet\sdk]
5.0.201 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 1.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Visual Studio 2019
Debugging with Edge v 89

[javiercn]

@dmanchura thanks for contacting us.

What's likely happening here is that the request content is being buffered by the HTTP client instance which internally does a fetch call to the underlying JS API.

Unfortunately I believe that is a browser limitation and you'll have to plan around it. Edge/Chrome added a feature in chrome 85 to support request streaming see here for details.

If you have such requirements I suggest for now you write a specialized component/method to deal with the file upload entirely on JavaScript.

[dmanchura]

@javiercn Thank you fore getting back to me.

Ok, got it, will work on an all JS solution for this part, thanks.