Uploading large file to SharePoint from io.BytesIO instance results in `io.UnsupportedOperation: fileno` exception

Question

I am using the Office365-REST-Python-Client library to upload some relatively large CSV files to a SharePoint document library via an io.BytesIO instance. I do this by passing the byte array to the following method:

from office365.sharepoint.folders.folder import Folder
from office365.sharepoint.files.file import File

def write_file_bytes(self, relative_url: str, file_name: str, file_bytes: bytes) -> None:

        folder: Folder = self.client_context.web.get_folder_by_server_relative_url(relative_url) 

        chunk_size: int = 1024 * 1024 * 15

        # File bytes to IO stream
        file_bytes: io.BytesIO = io.BytesIO(file_bytes)

        folder.files.create_upload_session(file_bytes, chunk_size=chunk_size, file_name=file_name).execute_query()

Based on this StackOverflow question, writing the file from a io.BytesIO instance is indeed possible, but the file_name and file_size should be passed as keyword arguments to chunk_uploaded. However, even in specifying a callback that takes the file size as an argument, I still get an io.UnsupportedOperation: fileno exception.

Uploading the file from either a byte array or an io.BytesIO instance is necessary due to the nature of what I am doing. So I can unfortunately not specify a local path to the file.

When performing a simple upload using the following:

folder.upload_file(file_name, file_bytes).execute_query()

Everything works as expected, but this is limited to a file size of 4.0MB, which is unfortunately too small for my needs.

Answer 1

My solution was to make use of a temp file and open it as a BufferedReader using a context manager, and write it to the relevant document library on SharePoint. This was implemented as follows:

def write_file_bytes(self, relative_url: str, file_name: str, file_bytes: bytes) -> None:

        folder: Folder = self.client_context.web.get_folder_by_server_relative_url(relative_url)

        chunk_size: int = 2048000

        # Check if the file already exists
        file: File = folder.files.get_by_url(file_name)

        if file.exists:

            # If the file exists, delete it
            file.delete_object().execute_query()

        # Create a temporary file and write the bytes to it
        with tempfile.NamedTemporaryFile(delete=False) as temp_file:
            temp_file.write(file_bytes)

        # Use the temporary file for uploading
        with open(temp_file.name, "rb") as file_to_upload:
            folder.files.create_upload_session(
                file=file_to_upload,
                chunk_size=chunk_size,
                file_name=file_name
            ).execute_query()