How to make sure that while will stop in every respect

Question

I'm writing a script for KODI and I have discovered an infinite loop when there is a code error. This loop means that I have to login to another account or restart my computer to stop KODI.

My question is therefore, how do I make sure that while will stop in every respect?

---

This is just a part of the script, and that the codes is wrapped in a try. You also need to know that the try always works if I raise an error.

The problem in code is the while part e.g. at p.get(). This results in an error because I have not checked whether the work is completed before I call get() (I have not correct this error for your sake).

The problem is that even though I use try, an unexpected error will cause the while to not stop!

def browse(separate, page):
    [...]
    # Getting meta data and subtitles
    pools = {'metadata': [], 'subtitles': []}
    with closing(multiprocessing.Pool(processes=2)) as pool:
        provider_meta = call_provider(PROVIDERS['meta_tmdb'])
        provider_subtitle = call_provider(PROVIDERS['subtitle_subscene'])
        for item in items:
            pools['metadata'].append(pool.apply_async(provider_meta.get, args=(item["info"]["code"], item["label"], item["info"]["year"]), callback=pool_stats))
            pools['subtitles'].append(pool.apply_async(provider_subtitle.get, args=(item["info"]["code"], item["label"], item["info"]["year"]), callback=pool_stats))
        pool_checklist = _create_checklist(pools)
        while not xbmc.abortRequested and not dialog.iscanceled():
            xbmc.sleep(100)
            # Check if are raise a error
            for p in pool_checklist:
                p.get()
            # Break when all requests are done 
            if all(p.ready() for p in pool_checklist):
                break
        else:
            return
    [...]

def _create_checklist(pools):
    plist = []
    for c in pools.values():
        for p in c:
            plist.append(p)
    return plist

---

UPDATE

I'm not sure whether the while only will be affected by the codes written in the while or whether there are other aspects that may affect the while process.

---

UPDATE 2

A test to see if get() returns an error. Tested with Python 3.x and not with Python 2.x which is what KODI is using.

---

UPDATE 3

A test to see if get() returns a raise error. Tested with Python 3.4.2 and Python 2.7.10.

from multiprocessing import Pool
from contextlib import closing
import time

def func(x, i):
    if i == 10:
        raise Exception('ttt')
    return {'x':i}

def go():
    try:
        def callback(x):
            print('done: '+str(x['x']))

        pools = []
        with closing(Pool(processes=2)) as pool:
            for i in range(20):
                pools.append(pool.apply_async(func, args=(i,i), callback=callback))
            while not all(p.ready() for p in pools):
                time.sleep(1)

        list = map(lambda p: p.get(), pools)
        for l in list:
            print(l)
        print('Finished with the script')
    except:
        print('ERROR')

if __name__ == '__main__':
    go()

---

UPDATE 4

The questions are still:

• Are the while only affected by the codes written in the while or can other aspects affect the while process?
• How do I make sure that the while will stop in every respect?

---

UPDATE 5

It does not appear that there is a solution where the while will stop in every respect. Therefore, I think that the simple solution would be to check the pools after the while.

def browse(separate, page):
    [...]
    # Getting meta data and subtitles
    pools = {'metadata': [], 'subtitles': []}
    with closing(multiprocessing.Pool(processes=2)) as pool:
        provider_meta = call_provider(PROVIDERS['meta_tmdb'])
        provider_subtitle = call_provider(PROVIDERS['subtitle_subscene'])
        for item in items:
            pools['metadata'].append(pool.apply_async(provider_meta.get, args=(item["info"]["code"], item["label"], item["info"]["year"]), callback=pool_stats))
            pools['subtitles'].append(pool.apply_async(provider_subtitle.get, args=(item["info"]["code"], item["label"], item["info"]["year"]), callback=pool_stats))
        pool_checklist = _create_checklist(pools)
        while not all(p.ready() for p in pool_checklist):
            if xbmc.abortRequested or dialog.iscanceled()
                return
            xbmc.sleep(100)
        # Check the pools for errors
        for p in pool_checklist:
            p.get()
    [...]

def _create_checklist(pools):
    plist = []
    for c in pools.values():
        for p in c:
            plist.append(p)
    return plist

Answer 1

The error occurred not at get(), but with target in apply_async(). This solution have therefore only made it possible to stop the process again.

The conclusion being that. The loop will continue although there is a fault with Pool, and therefore you can always exit the loop.

def browse(separate, page):
    [...]
    # Getting meta data and subtitles
    pools = {'metadata': [], 'subtitles': []}
    with closing(multiprocessing.Pool(processes=2)) as pool:
        provider_meta = call_provider(PROVIDERS['meta_tmdb'])
        provider_subtitle = call_provider(PROVIDERS['subtitle_subscene'])
        for item in items:
            pools['metadata'].append(pool.apply_async(provider_meta.get, args=(item["info"]["code"], item["label"], item["info"]["year"]), callback=pool_stats))
            pools['subtitles'].append(pool.apply_async(provider_subtitle.get, args=(item["info"]["code"], item["label"], item["info"]["year"]), callback=pool_stats))
        pool_checklist = _create_checklist(pools)
        # Loop
        while not all(p.ready() for p in pool_checklist):
            if xbmc.abortRequested or dialog.iscanceled():
                return
            xbmc.sleep(100)
        # Check the results for errors
        for p in pool_checklist:
            p.get()
    [...]

Furthermore be noted for Python 3.4.2 and 2.7.10, that Pool will only raise an error when it is completed, and any additional can get() be used to check the results for errors.