Is there a way to configure the retention period for google cloud pub/sub?

7.9k views Asked by At

Is there a way to specify the retention period for the messages in google cloud pub/sub? Let's say I have a subscriber started consuming messages, but the service crashed and is back online a day later after the bug is fixed. Can it continue from the last checkpoint? How soon the consumer needs to reconnect with the pub/sub to guarantee that it won't lose any messages?

On a similar note, the Amazon Kinesis can keep messages up to 24 hours, and Kafka can have retention policy based on time or/and disk space.

2

There are 2 answers

2
Takashi Matsuo On BEST ANSWER

Subscriptions can retain undelivered messages for up to 7 days. It's not configurable.

FYI various limits including this one are described here: https://cloud.google.com/pubsub/quotas

0
Patrick Decat On

Google Cloud PubSub Subscriptions can now be created with a custom message retention duration using the API and gcloud CLI, e.g.:

# gcloud pubsub subscriptions create mysub --topic=mytopic --message-retention-duration=48h

Accepted values are between 10 minutes to 7 days (the default).

For reference: https://cloud.google.com/sdk/gcloud/reference/pubsub/subscriptions/create

Subscriptions can now also be updated: https://cloud.google.com/sdk/gcloud/reference/pubsub/subscriptions/update