Uploading ReadableStream as part of FormData from Browser

1.7k views Asked by At

On my Create-React-App application, users can upload a file of potentially large sizes. From these files, I chunk and encrypt them (done in the client for transparency) while in the form of a stream. I'm now hoping to send this stream to an external API, that accepts a FormData parameter, i.e. file=FILE.

I'm currently trying using the fetch function to do this, but FormData.append() doesn't accept a stream, only a string or a file (which is not an option because the file may be huge, >1GB).

Is there any other way I can do this, other than sending it to my server to send to the API? Really appreciate any thoughts/help!

If relevant, this is the code i'm using now to perform the fetch:

export const addFilesToDataset = async (result: TransformStream): Promise<void> => {
    // ...
    const formData = new FormData();
    formData.append('file', result); // <- this won't work
    const respData = await fetch(url, {
        method: 'POST',
        body: result.readable,
    });
    const jsonData = await respData.json();
    console.log('Res: ', jsonData);
};

EDIT:

For anyone interested, turns out, creating an ArrayBuffer using the result stream doesn't crash the page, and profiling it doesn't show a significant surge in memory, for some reason.

Not too sure why, maybe the browser is able to temporarily create a file in storage to reduce memory usage? Appreciate if anyone can provide an answer!

EDIT 2:

Turns out the previous answer seems to be misleading; seeing the browser task manager (I'm using Edge) shows that the entire file is actually in memory, so it seems i don't have the answer yet lol

1

There are 1 answers

1
selcuk On
export const addFilesToDataset = async (result: TransformStream): Promise<void> => {
    // ...
    const formData = {
      "file": result
    }
    
    const respData = await fetch(url, {
        method: 'POST',
        body: result.readable,
    });
    const jsonData = await respData.json();
    console.log('Res: ', jsonData);
};

try this