I am running Celery+Kombu 4.4.6 on AWS SQS and want to revoke and terminate tasks.
Reading through documentation and SO posts, the transport needs to allow broadcast messages. SQS does not do broadcast messages and Celery+Kombu needs to use SimpleDB for those. That option was turned off by default long way back in version 1.x. To enables it, support_fanout = True needs to be added to the transport options.
But adding just that option is not working for me and I can't figure out what am I missing. Possible options are:
- SimpleDB - it is not clear to me how do I even enable SimpleDB. I do see documentation in AWS, but I do not see it as a separate service.
- Any additional config to be added?
- Looking briefly at the SQS code, seems like SimpleDB is the only option for this. Is that correct?
- Any other option to enable task revocation on SQS?
In my app.celery I have:
app = Celery('app',
broker=''sqs://<Access key>:<secret key>@')),
backend='cache+memcached://<host>:11211/')),
)
And in my app.settings I have:
CELERY_BROKER_URL='sqs://<access key>:<secret key>@'))
CELERY_BROKER_TRANSPORT_OPTIONS = {
'region': '<region>',
'supports_fanout': True,
}
CELERY_DEFAULT_QUEUE = 'app'
CELERY_DEFAULT_EXCHANGE = 'app'
CELERY_DEFAULT_ROUTING_KEY = 'app'
My final solution was to use Amazon MQ with a RabbitMQ instance. Amazon SimpleDB seems to be gone, making any support in Celery+Kombu obsolete and broken.