Timeout on a Python function call inside timeit

Question

I have a function, let's call it my_function(*args, **kwargs), that depending on the arguments passed to it takes anywhere from 30 seconds to many many hours (days). I have a list of different arguments and I want to know how long the functions will take given the arguments.

I am still very new to using timeit, but I have learn enough to do this part; however, this wouldn't be the very efficient for my purposes. Any set of arguments that take longer than 4 hours to complete are consider intractable, despite all of them being able to be solved in "finite" time. As some (possibly most) of the arguments will results in run times that will take upwards of 20+ hours, I am looking for a way to terminate a test after 4 hours so that I don't have to waste time after figuring out that it is intractable.

I have looked at Timeout on a Python function call and Stop code after time period, and this might be close enough of a question to be a duplicate, but I am having trouble integrating those answers with timeit so that times less that 4 hours are recorded like they should be while long runs return some valid time greater than 4 hours.

What would be the best way to go about doing this?

EDIT : One of the problems that I have had with it is that the answers that I've seen have take in func(*args,**kwargs) while the timeit functions looks like this:

timeit.Timer('my_function(*args, **kwargs)', setup=setup).timeit(1)

I don't know how to handle this form.

EDIT : The original answer that I provided below using threads doesn't actually terminate the threads. This can easily be shown by running it with the following function.

def foo(x):
    for i in xrange(1, x + 1):
        sleep(1)
        print i
    return x

Using a code that involves multiprocessing.Pool, which actually has a terminate(), allow for this.

Answer 1

Based on the answer found in Timeout function using threading in python does not work. If you try it out on foo(x) it does indeed stop counting unlike the my previous answer.

import multiprocessing as mp
import timeit

def timeout(func, args=(), kwargs=None, TIMEOUT=10, default=None, err=.05):

    if hasattr(args, "__iter__") and not isinstance(args, basestring):
        args = args
    else:
        args = [args]    
    kwargs = {} if kwargs is None else kwargs

    pool = mp.Pool(processes = 1)

    try:
        result = pool.apply_async(func, args=args, kwds=kwargs)
        val = result.get(timeout = TIMEOUT * (1 + err))
    except mp.TimeoutError:
        pool.terminate()
        return default
    else:
        pool.close()
        pool.join()
        return val

def Timeit(command, setup=''):
    return timeit.Timer(command, setup=setup).timeit(1)

def timeit_timeout(command, setup='', TIMEOUT=10, default=None, err=.05):
    return timeout(Timeit, args=command, kwargs={'setup':setup},
                   TIMEOUT=TIMEOUT, default=default, err=err) 

Answer 2

After some more fiddling I have a initial answer based on Timeout function using threading. I would still love to hear from anyone who has better ideas as I am still new to this.

def timeout(func, args=None, kwargs=None, TIMEOUT=10, default=None, err=.05):
    if args is None:
        args = []
    elif hasattr(args, "__iter__") and not isinstance(args, basestring):
        args = args
    else:
        args = [args]

    kwargs = {} if kwargs is None else kwargs

    import threading
    class InterruptableThread(threading.Thread):
        def __init__(self):
            threading.Thread.__init__(self)
            self.result = None

        def run(self):
            try:
                self.result = func(*args, **kwargs)
            except:
                self.result = default

    it = InterruptableThread()
    it.start()
    it.join(TIMEOUT* (1 + err))
    if it.isAlive():
        return default
    else:
        return it.result

def timeit_timeout(command, setup='', TIMEOUT=10, default=None, err=.05):
    import timeit
    f = lambda: timeit.Timer(command, setup=setup).timeit(1)
    return timeout(f,TIMEOUT=TIMEOUT, default=default, err=err) 

if __name__ == '__main__':    
    from time import sleep
    setup = 'gc.enable()\nfrom time import sleep'
    for n in range(1,15):
        command = 'sleep({})'.format(n)
        print timeit_timeout(command, setup=setup, default=float('Inf'))