How can I interrupt or send signals to python Threads without cooperation from the thread?

Question

Previously I thought this wasn't possible to non-cooperatively interrupt a Thread in Python. A example might be the main thread asking to immediately cancel other theads in the process. But, from the signal docs:

Python signal handlers are always executed in the main Python thread of the main interpreter, even if the signal was received in another thread. This means that signals can't be used as a means of inter-thread communication.

... however ...

https://youtu.be/9OOJcTp8dqE?si=_GVmIcCGX9JTawXZ&t=1872

suggests it is:

[…] Python already had a mechanism for requesting threads to handle asynchronous signals and exceptions […]

I see Java has Thread.interrupt(), but not Python.

So, is it possible now in Python or not?

Answer 1

TL;DR: There is no way to do that!

Detailing this:

How exactly would you want threads to respond "non colaboratively"?

The mechanism mentioned in the video, btw, is an internal mechanism to be used by the Python runtime itself, in hacks such as the attempted in the lecture.

So, back to the how it should work - a signal handler is a registred callback, so even if this would be exposed for Python code - a per-thread signal callback, making this work would be much harder than simply have a queue from where to read instructions from. (and registering a per-thread callback can hardly be said to be "non collaborative" on the thread side).

Anyway, no. Once a Python thread is running, it will keep running until it is "out of code" (it returns from the entry function passed as "target" argument when starting the thread). One can force this to happen by consuming elements from a Queue for inter-thread communication, and given a proper signal on the queue, raise an exception, or break out of a worker-loop. (the pattern known as "poison pill"). That is just how it works.

And in Python 3.12, where one can make use of "subinterpreters" that are GIL free, the same logic applies - but that the entry point, instead of being a function is a string of code: if the code evaluated from the string "ends", the interpreter goes idle - and one has to explicitly shut it down. But there are no mechanisms other than shutting down an idle subinterpreter that are meant to be used. (Although one can make a call to shutdown a non-idle subinterpreter, that will more likely than not lead to a Python runtime crash).