Upload large files from the Workbook Object to the IBM Cloud Object Storage using the Java

410 views Asked by At

I am creating a large excel file and while uploading it using the Java SDK for the IBM Cloud Object Storage I getting the following

com.ibm.cloud.objectstorage.services.s3.model.AmazonS3Exception: Request Entity Too Large (Service: Amazon S3; Status Code: 413; Error Code: 413 Request Entity Too Large; Request ID: null)

What I am doing? I had created an excel file using the Apache POI library and Now I want to transfer this object output stream to the IBM COS bucket. How Can I achieve this ?

Code Using stream

public boolean uploadWorkbookObjectToBucket(Workbook wb, String fileName) {
    try {
        // converting the workbook to serialized output stream 
        ByteArrayOutputStream theBytes = new ByteArrayOutputStream(); // create a new output stream to store the object data
        ObjectOutputStream serializer = new ObjectOutputStream(theBytes); // set the object data to be serialized
        wb.write(serializer); // serialize the object data
        serializer.flush(); // flushing
        serializer.close(); // closing the serializer
        InputStream stream = new ByteArrayInputStream(theBytes.toByteArray()); // convert the serialized data to a new input stream to store
        
        // creating the metadata to upload the data
        ObjectMetadata metadata = new ObjectMetadata(); // define the metadata
        //metadata.setContentType("application/x-java-serialized-object"); // set the metadata
        //metadata.setContentLength(theBytes.size()); // set metadata for the length of the data stream
    
        PutObjectResult result = cos.putObject(bucketName, fileName, stream, metadata);
        
        if(result != null) {
            return true;
        }
        
    } catch(Exception e) {
        logger.error("Exception while uploading file to bucket using the workbook", e);
    }
    return false;
}

Code Using Transfer Manager

private TransferManager getTransferManager() {
    return  TransferManagerBuilder.standard()
            .withS3Client(cos)
            .withMinimumUploadPartSize(partSize)
            .withMultipartCopyThreshold(thresholdSize)
            .build();
}
public boolean uploadFileUsingTransferManager(Workbook wb, String fileName) throws Exception {
    
    // converting the workbook to serialized output stream 
    ByteArrayOutputStream theBytes = new ByteArrayOutputStream(); // create a new output stream to store the object data
    ObjectOutputStream serializer = new ObjectOutputStream(theBytes); // set the object data to be serialized
    wb.write(serializer); // serialize the object data
    serializer.flush(); // flushing
    serializer.close(); // closing the serializer
    InputStream stream = new ByteArrayInputStream(theBytes.toByteArray()); // convert the serialized data to a new input stream to store
    
    // creating the metadata to upload the data
    ObjectMetadata metadata = new ObjectMetadata(); // define the metadata
    // metadata.setContentType("application/x-java-serialized-object"); // set the metadata
    // metadata.setContentLength(theBytes.size()); // set metadata for the length of the data stream
    
    TransferManager transferManager = getTransferManager();
    
    try {
        Upload transfer = transferManager.upload(bucketName, fileName, stream, metadata);
        transfer.waitForCompletion();
        // transfer.addProgressListener(new P);
        return true;
    } catch(Exception e) {
        logger.error("Exception while uploading file to bucket using the workbook", e);
    } finally {
        transferManager.shutdownNow();
        wb.close();
    }
    
    
    return false;
}
0

There are 0 answers