Why does timeit report slower code as faster one?

318 views Asked by At

I tested this with python 3.5 in Debian Stretch.
I tried benchmark the "Avoiding dots" optimization.
As expected, the "Avoiding dots" optimization is really much faster.
Unexpected, timeit reports the slower code as the faster. What is the reason?

$ time python3 -m timeit -s "s=''" "s.isalpha()"
10000000 loops, best of 3: 0.119 usec per loop

real    0m5.023s
user    0m4.922s
sys 0m0.012s
$ time python3 -m timeit -s "isalpha=str.isalpha;s=''" "isalpha(s)"
1000000 loops, best of 3: 0.212 usec per loop

real    0m0.937s
user    0m0.927s
sys 0m0.000s
2

There are 2 answers

2
Davis Herring On BEST ANSWER

timeit did 10 times as many iterations in the “slow” case. It adaptively tries more iterations to find a number that balances statistical quality and waiting time.

0
illiterate On

Thank to Davis Herring's answer.
Let's make understand at more details:
From python3 -m timeit -h:

If -n is not given, a suitable number of loops is calculated by trying
successive powers of 10 until the total time is at least 0.2 seconds.

Verify by calculate the details information:

$ time python3 -m timeit -v -s "s=''" "s.isalpha()"
10 loops -> 3.44e-06 secs
100 loops -> 1.29e-05 secs
1000 loops -> 0.000117 secs
10000 loops -> 0.00116 secs
100000 loops -> 0.0118 secs
1000000 loops -> 0.116 secs
10000000 loops -> 1.17 secs
raw times: 1.21 1.21 1.21
10000000 loops, best of 3: 0.121 usec per loop

real    0m4.992s
user    0m4.977s
sys 0m0.012s

All the x loops -> y secs time sum is used to determine the suitable loops number.
Each items in the "raw time" line are single repeat timer result(the -r option determine the number of items in the "raw time" line).
Almost all time is matched:

>>> 3.44e-06+1.29e-05+0.000117+0.00116+0.0118+0.116+1.17+1.21+1.21+1.21
4.92909334
>>> 4.992-4.92909334
0.06290666000000034