Azure Blob Async copy delete java

Question

I am new to Azure Blob, I am trying to copy files in a folder and once the copy is done, delete the files in Async. I can list the files and URL but copy and delete fails.

What am I missing here.

 BlobServiceClientBuilder blobServiceClientBuilder = new BlobServiceClientBuilder()
            .endpoint(endpoint)
            .sasToken(sharedAccessSignatures);

    ExecutorService executorService = Executors.newFixedThreadPool(10);

    try {
        // List files in the source folder
        List<String> sourceFiles = listFiles(blobServiceClientBuilder.buildClient(), sourceFolderPath, containerName);

        // Check if there are files to process
        if (sourceFiles.isEmpty()) {
            System.out.println("No files found in the source folder. Exiting.");
            return;
        }

        // Asynchronously copy and delete each file
        List<CompletableFuture<Void>> futures = sourceFiles.stream()
                .map(file -> CompletableFuture.runAsync(() -> copyAndDeleteFile(blobServiceClientBuilder.buildClient(), file), executorService))
                .collect(Collectors.toList());

        // Wait for all asynchronous operations to complete
        CompletableFuture<Void> allOf = CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]));
        allOf.join();
    } finally {
        // Shutdown the executor service
        executorService.shutdown();
    }

    System.out.println("All copy operations completed. Exiting.");
}

private static List<String> listFiles(BlobServiceClient blobServiceClient, String folderPath, String containerName) {
    try {
        List<String> filePaths = blobServiceClient.getBlobContainerClient(containerName)
                .listBlobs()
                .stream()
                .filter(blobItem -> !blobItem.getName().endsWith("/") && blobItem.getName().startsWith(folderPath))
                .map(blobItem -> blobItem.getName())
                .collect(Collectors.toList());

        if (filePaths.isEmpty()) {
            System.out.println("No files found in the source folder: " + folderPath);
        } else {
            System.out.println("File Paths: " + filePaths);
        }

        return filePaths;
    } catch (Exception e) {
        System.err.println("Error listing files: " + e.getMessage());
        return Collections.emptyList();
    }
}

private static void copyAndDeleteFile(BlobServiceClient blobServiceClient, String filePath) {
    try {
        String fileName = Paths.get(filePath).getFileName().toString();
        System.out.println("FileName: " + fileName);

        String sourceBlobName = sourceFolderPath + "/" + fileName;
        String destBlobName = destFolderPath + "/" + fileName;

        System.out.println("SourceBlobName: " + sourceBlobName);
        System.out.println("DestinationBlobName: " + destBlobName);

        var sourceBlobClient = blobServiceClient.getBlobContainerClient(containerName)
                .getBlobClient(sourceBlobName);
        var destBlobClient = blobServiceClient.getBlobContainerClient(containerName)
                .getBlobClient(destBlobName);

        System.out.println("Source Blob URL: " + sourceBlobClient.getBlobUrl());
        System.out.println("Destination Blob URL: " + destBlobClient.getBlobUrl());

        // Ensure source blob exists before copying
        if (!sourceBlobClient.exists()) {
            throw new RuntimeException("Source blob does not exist: " + sourceBlobName);
        }

        // Copy the file asynchronously
        CopyStatusType copyInfo = ((BlobCopyInfo) destBlobClient.beginCopy(sourceBlobClient.getBlobUrl(), null)).getCopyStatus();

        // Wait for the copy to complete
        while (copyInfo.equals(CopyStatusType.PENDING)) {
            Thread.sleep(1000);
            copyInfo = destBlobClient.getProperties().getCopyStatus();
        }

        // Check if the copy was successful
        if (copyInfo.equals(CopyStatusType.SUCCESS)) {
            // Delete the source blob
            sourceBlobClient.delete();
            System.out.println("Blob copy and delete completed successfully.");
        } else {
            throw new RuntimeException("Copy operation failed.");
        }

    } catch (Exception e) {
        System.err.println("Error copying and deleting file: " + e.getMessage());
    }
}

Error:

Error copying and deleting file: Status code 401, "CannotVerifyCopySourceServer failed to authenticate the request. Please refer to the information in the www-authenticate header. RequestId:c902d7e8-201e-004b-2954-2d97e4000000 Time:2023-12-12T23:37:19.1311821Z"

Answer 1

Error copying and deleting file: Status code 401, "CannotVerifyCopySourceServer failed to authenticate the request. Please refer to the information in the www-authenticate header RequestId:c902d7e8-201e-004b-2954-2d97e4000000 Time:2023-12-12T23:37:19.1311821Z"

The above error occurs when you don't have proper permission in the SAS token to delete the source folder.

In my environment, I have two folders named sample and demo stored in my Azure Blob storage

Generated sas token through the portal with permission read,write,list and delete.

Portal:

Now, Using the below code I can able to copy from the source folder[sample] to the destination folder[demo] and also able to delete the folder.

Code:

import com.azure.storage.blob.BlobServiceClient;
import com.azure.storage.blob.BlobServiceClientBuilder;
import com.azure.storage.blob.models.CopyStatusType;
import java.nio.file.Paths;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.Collectors;

public class App {
    private static final String endpoint = "https://xxxxx.blob.core.windows.net/";
    private static final String sharedAccessSignatures = "sv=2022-11-02&ss=bfqt&srt=co&sp=rwdltfx&se=2023-12-13T13:24:35Z&st=2023-12-13T05:24:35Z&spr=https&sig=xxxxxx";
    private static final String containerName = "test";
    private static final String sourceFolderPath = "sample";
    private static final String destFolderPath = "demo";

    public static void main(String[] args) {
        BlobServiceClientBuilder blobServiceClientBuilder = new BlobServiceClientBuilder()
                .endpoint(endpoint)
                .sasToken(sharedAccessSignatures);

        ExecutorService executorService = Executors.newFixedThreadPool(10);

        try {
            // List files in the source folder
            List<String> sourceFiles = listFiles(blobServiceClientBuilder.buildClient(), sourceFolderPath, containerName);

            // Check if there are files to process
            if (sourceFiles.isEmpty()) {
                System.out.println("No files found in the source folder. Exiting.");
                return;
            }

            // Asynchronously copy and delete each file
            List<CompletableFuture<Void>> futures = sourceFiles.stream()
                    .map(file -> CompletableFuture.runAsync(() -> copyAndDeleteFile(blobServiceClientBuilder.buildClient(), file), executorService))
                    .collect(Collectors.toList());

            // Wait for all asynchronous operations to complete
            CompletableFuture<Void> allOf = CompletableFuture.allOf(futures.toArray(new CompletableFuture[0]));
            allOf.join();
        } finally {
            // Shutdown the executor service
            executorService.shutdown();
        }

        System.out.println("All copy operations completed. Exiting.");
    }

    private static List<String> listFiles(BlobServiceClient blobServiceClient, String folderPath, String containerName) {
        try {
            List<String> filePaths = blobServiceClient.getBlobContainerClient(containerName)
                    .listBlobs()
                    .stream()
                    .filter(blobItem -> !blobItem.getName().endsWith("/") && blobItem.getName().startsWith(folderPath))
                    .map(blobItem -> blobItem.getName())
                    .collect(Collectors.toList());

            if (filePaths.isEmpty()) {
                System.out.println("No files found in the source folder: " + folderPath);
            } else {
                System.out.println("File Paths: " + filePaths);
            }

            return filePaths;
        } catch (Exception e) {
            System.err.println("Error listing files: " + e.getMessage());
            return Collections.emptyList();
        }
    }

    private static void copyAndDeleteFile(BlobServiceClient blobServiceClient, String filePath) {
    try {
        String fileName = Paths.get(filePath).getFileName().toString();
        System.out.println("FileName: " + fileName);

        String sourceBlobName = sourceFolderPath + "/" + fileName;
        String destBlobName = destFolderPath + "/" + fileName;

        System.out.println("SourceBlobName: " + sourceBlobName);
        System.out.println("DestinationBlobName: " + destBlobName);

        var sourceBlobClient = blobServiceClient.getBlobContainerClient(containerName)
                .getBlobClient(sourceBlobName);
        var destBlobClient = blobServiceClient.getBlobContainerClient(containerName)
                .getBlobClient(destBlobName);

        System.out.println("Source Blob URL: " + sourceBlobClient.getBlobUrl());
        System.out.println("Destination Blob URL: " + destBlobClient.getBlobUrl());

        // Ensure source blob exists before copying
        if (!sourceBlobClient.exists()) {
            throw new RuntimeException("Source blob does not exist: " + sourceBlobName);
        }

        // Copy the file asynchronously
        String sourceBlobUrl = sourceBlobClient.getBlobUrl() + "?" + sharedAccessSignatures;
        var syncPoller = destBlobClient.beginCopy(sourceBlobUrl, null);

        // Wait for the copy to complete
        syncPoller.waitForCompletion();

        // Get the copy status after completion
        CopyStatusType copyStatus = syncPoller.poll().getValue().getCopyStatus();
        System.out.println("Copy Status: " + copyStatus);

        // Check if the copy was successful
        if (copyStatus.equals(CopyStatusType.SUCCESS)) {
            // Delete the source blob
            sourceBlobClient.delete();
            System.out.println("Blob copy and delete completed successfully.");
        } else {
            throw new RuntimeException("Copy operation failed.");
        }

    } catch (Exception e) {
        System.err.println("Error copying and deleting file: " + e.getMessage());
        e.printStackTrace(); 
    }
}
}

Output:

File Paths: [sample/Animation.gif, sample/Animation1.gif, sample/csvanimation1.gif]
FileName: csvanimation1.gif
FileName: Animation1.gif
SourceBlobName: sample/csvanimation1.gif
FileName: Animation.gif
SourceBlobName: sample/Animation.gif
DestinationBlobName: demo/Animation.gif
DestinationBlobName: demo/csvanimation1.gif
SourceBlobName: sample/Animation1.gif
DestinationBlobName: demo/Animation1.gif
Source Blob URL: https://venkat123.blob.core.windows.net/test/sample%2FAnimation.gif
Destination Blob URL: https://venkat123.blob.core.windows.net/test/demo%2FAnimation.gif
Source Blob URL: https://venkat123.blob.core.windows.net/test/sample%2Fcsvanimation1.gif
Source Blob URL: https://venkat123.blob.core.windows.net/test/sample%2FAnimation1.gif
Destination Blob URL: https://venkat123.blob.core.windows.net/test/demo%2Fcsvanimation1.gif
Destination Blob URL: https://venkat123.blob.core.windows.net/test/demo%2FAnimation1.gif
Copy Status: success
Blob copy and delete completed successfully.
Copy Status: success
Copy Status: success
Blob copy and delete completed successfully.
Blob copy and delete completed successfully.
All copy operations completed. Exiting.

Portal: