Performance of File Hash algorithms

853 views Asked by At

In my Perl application, I need compare two versions of a file and detect whether they have changed.

I'm trying to choose between MD5 or SHA. This is not about security. This is about fastest way to compare files. I'm inclined towards MD5.

However, when I ran benchmarks, it suggests otherwise.

Any recommendations?

Here's a benchmark I ran with the largest file in my app.

Benchmark: timing 10000000 iterations of MD5, SHA...
   MD5: -0.199009 wallclock secs ( 0.07 usr +  0.01 sys =  0.08 CPU) @ 125000000.00/s (n=10000000)
        (warning: too few iterations for a reliable count)
   SHA: 0.494412 wallclock secs ( 0.06 usr +  0.00 sys =  0.06 CPU) @ 166666666.67/s (n=10000000)
        (warning: too few iterations for a reliable count)
       Rate  MD5  SHA
MD5 125000000/s   -- -25%
SHA 166666667/s  33%   --
1

There are 1 answers

2
Joni On

MD5 may be faster to compute than SHA1 because of its simpler structure. Then again, getting the data from the disk will be slower than keeping MD5 or SHA1 checksum up to date, so it will not really matter in practice.