Can't create Race Condition in Python 3.11 using multiple threads

Question

I believe this is a difference in Python 3.10 and above from older versions. Could someone explain this?

import threading
import time

counter = 0

lock = threading.Lock()


def increment():
    global counter

    for _ in range(10**6):
        counter += 1


threads = []
for i in range(4):
    x = threading.Thread(target=increment)
    threads.append(x)

for t in threads:
    t.start()

for t in threads:
    t.join()

print(counter)

Why does this code does not produce a race condition in Python 3.11?

However, when I change this line to

counter += int(1)

then the race condition occurs.

Answer 1

Based on https://old.reddit.com/r/learnprogramming/comments/16mlz4h/race_condition_doesnt_happen_from_python_310/k198umz/:

In 3.10 an optimisation (introduced by commit https://github.com/python/cpython/commit/4958f5d69dd2bf86866c43491caf72f774ddec97 ) made it instead release and acquire the GIL only at specific bytecode instructions, rather than at any of them. In your example, your code is a loop of [LOAD_GLOBAL counter, LOAD_CONST 1, INPLACE_ADD, STORE_GLOBAL counter], importantly None of these are magic "eval breaking" instructions that check the GIL (the final JUMP_ABSOLUTE however is, and so each iteration of the loop is a potential GIL release+acquire point). That means that the load of counter, adding 1, and storing it back, all happen atomically because there were no bytecode instructions in this sequence that caused the eval breaker to do its periodic GIL release+acquire cycle, meaning the GIL is held the entire time. This explains the behaviour you see.

For example, the CALL_FUNCTION bytecode instruction is one of these magic eval breakers, so changing your code to have a def one(): return 1 and counter += one() will cause the original race to return.

In this case you can check the bytecode translation using dis and notice the CALL_FUNCTION bytecode instruction:

import dis

def increment1():
    global counter
    for _ in range(10**6):
        counter += 1

def increment2():
    global counter
    for _ in range(10**6):
        counter += int(1)

dis.dis(increment1)
dis.dis(increment2)

Answer 2

PROBLEM:

There are problems loading your counter counter += int(1). The problem is related to bytecode, because there are no bytcode instructions in your sequence to execute the loop,

SOLUTION:

To create Race Condition in Python 3.11, you need to use a disassembler for Python: dis — Disassembler for Python bytecode. You can import dis, because a general disassembly is necessary, then notice increment_n_1 and increment_n_2 bytecode. Dis is a built-in module that provides tools for disassembling Python bytecode. Bytecode is a low-level intermediate representation of Python code that is produced by the Python compiler and executed by the Python Virtual Machine.

The first increment will be:

def increment_n_1():
     global counter
     for x in range(10**6):
         counter += 1

The second increment will be:

def increment_n_2():
     global counter
     for x in range(10**6):
         counter += int(1)

Finally you need to use dis:

incr1 = dis.dis(increment_n_1)
incr2 = dis.dis(increment_n_2)

Complete Code

import dis

#First increment
def increment_n_1():
    global counter
    for x in range(10**6):
        counter += 1

#Second increment
def increment_n_2():
    global counter
    for x in range(10**6):
        counter += int(1)

incr1 = dis.dis(increment_n_1)
incr2 = dis.dis(increment_n_2)

Output:

  7           0 LOAD_GLOBAL              0 (range)
              2 LOAD_CONST               1 (1000000)
              4 CALL_FUNCTION            1
              6 GET_ITER
        >>    8 FOR_ITER                 6 (to 22)
             10 STORE_FAST               0 (x)

  8          12 LOAD_GLOBAL              1 (counter)
             14 LOAD_CONST               2 (1)
             16 INPLACE_ADD
             18 STORE_GLOBAL             1 (counter)
             20 JUMP_ABSOLUTE            4 (to 8)

  7     >>   22 LOAD_CONST               0 (None)
             24 RETURN_VALUE
 12           0 LOAD_GLOBAL              0 (range)
              2 LOAD_CONST               1 (1000000)
              4 CALL_FUNCTION            1
              6 GET_ITER
        >>    8 FOR_ITER                 8 (to 26)
             10 STORE_FAST               0 (x)

 13          12 LOAD_GLOBAL              1 (counter)
             14 LOAD_GLOBAL              2 (int)
             16 LOAD_CONST               2 (1)
             18 CALL_FUNCTION            1
             20 INPLACE_ADD
             22 STORE_GLOBAL             1 (counter)
             24 JUMP_ABSOLUTE            4 (to 8)

 12     >>   26 LOAD_CONST               0 (None)
             28 RETURN_VALUE