Is this the proper way to use the timeit module?

Question

I have the following code and results.

Tests 1 and 2 are identical (at least I think they are) but they always take a different amount of time to run.

My questions are:

1. Am I using timeit() correctly?
2. Do milliseconds matter when evaluating one syntax vs another?
3. If I am not doing anything wrong, why the difference?
4. Is timeit the correct way to test performance or are there better ways?

Code:

import timeit

def test1(a="String1", b="String2", c="String3",
          d="I took a heavenly ride through our silence "):
    r = a + b + c + d

def test2(e="String1", f="String2", g="String3",
          h="I took a heavenly ride through our silence "):
    s = e + f + g + h

def test3(e="String1", f="String2", g="String3",
          h="I took a heavenly ride through our silence "):
    t = e.join([f,g,h])


if __name__ == '__main__':
    reps = 10000000

    print("Test 1")
    print(timeit.timeit("test1()", number=reps,
                        setup="from __main__ import test1"))
    print("Test 2")
    print(timeit.timeit("test2()", number=reps,
                        setup="from __main__ import test2"))
    print("Test 3")
    print(timeit.timeit("test3()", number=reps,
                        setup="from __main__ import test3"))

    print(" ")
    print(" ")

    print("Test 2")
    print(timeit.timeit("test2()", number=reps,
                        setup="from __main__ import test2"))
    print("Test 1")
    print(timeit.timeit("test1()", number=reps,
                        setup="from __main__ import test1"))
    print("Test 3")
    print(timeit.timeit("test3()", number=reps,
                        setup="from __main__ import test3"))

Results:

Test 1
3.46861560261924
Test 2
3.4760945739099185
Test 3
6.51532737832723


Test 2
3.475778266347378
Test 1
3.465791808905923
Test 3
6.493744207694682

Answer 1

Documentation link.

• Processes running on the system may interfere with the timing as timeit measures clock time, not CPU time.

• By default, timeit() temporarily turns off garbage collection during the timing. The advantage of this approach is that it makes independent timings more comparable. This disadvantage is that GC may be an important component of the performance of the function being measured.

To call timeit() repeatedly,

def test(): L = [] for i in range(10): L.append(i)

if __name__ == '__main__': import timeit print(timeit.repeat("test()", setup="from __main__ import test"))