How to speed up shutil.copy()?

Question

Here I'm using the code in python like:

if option2 == 1:
    try:
        global option2, Nimages

        for sur in fm_path:
            shutil.copy(sur,file1) # here how to speed up the copy function

        option2 = 0
    except shutil.Error as e:
        print('Error: %s' % e)

While executing this, if I copy a lengthy file it took too much time to copy. Any suggestion to reduce the time?

Answer 1

If you are dealing with files with big size, you could try to do file open/write as a copy. As mentioned in Faster Python File Copy, the default buffer size in shutil.copyfileobj (which is the real function doing file-copy in shutil) is 16*1024, 16384. You can use bigger buffer size then.

ex,

with open(src, 'rb') as fin:
    with open(dst, 'wb') as fout:
        shutil.copyfileobj(fin, fout, 128*1024)

Answer 2

Starting with Python 3.8, shutils offers platform-specific faster alternatives to copy files. Read more about it here:

• https://docs.python.org/3/library/shutil.html#shutil-platform-dependent-efficient-copy-operations