I am facing the problem of slow deletes and downloads using Cyberduck (or similar Openstack Browsers such as cloudberry) when accessing large files that were chunked into small parts. See for Opentack Swift chunking link. The large files, are considered to have a file size between 1 and 10 GB using the segment size of 10mb. By slow I mean it takes over an hour for a 10GB file to identify all the chunks before it will start downloading. It all runs on the openstack environment of a national reputable cloud provider. Once it started downloading, I reach a normal rate of 2 to 10 MB per second, which is acceptable for our connection. The inconvenience is that I am not able to adjust the segment size as within Cloudberry Drive which is responsible for the chunking, as this is not yet possible. I thought of changing clients that do support adjusting the chunking size, but I only found cloudberry to fit my other needs.
- So first, is this slow performance caused by the chunking of files?
- Second, are there ways of tuning this performance maybe on the side of openstack?
The issue was related to the stack provider, as of networking problems all deletes on the openstack cloud were slow. As the file consisted of many chuncks, the deletion of the whole file became extremely slow.