I have a celery tasks that generates a list of items, specifically a list of slices from an image, and I want to be able to start a task to process each of the generated slices. Assume I have something similar to this:
@app.task()
def slice(img_path, width, height):
result = []
img = Image.open(img)
for box in get_slice_boxes(width, height):
result.append(img.crop(box))
return result
@app.task()
def process(img_slice):
# do something with the image slice
I've tried using a chord with a .map but as far as I understand I would need to have the list of items at the moment of the task instantiation which would require to run the slice task in the foreground. What I'm aiming for is to slice the image in the background and start one process task for each.
Is this even possible to do with celery? Is there a way I can achieve a similar result with something besides celery?