How can I automatically expire messages on an Azure Service Bus subscription?

Question

The documentation on BrokeredMessage says:

If the TTL set on a message by the sender exceeds the destination's TTL, then the message's TTL will be overwritten by the later value. See DefaultMessageTimeToLive, DefaultMessageTimeToLive and DefaultMessageTimeToLive to learn more about how to control message TTL at an entity level.

So, to achieve messages to be automatically removed from my "Log" subscription, When creating the subscription, I have set the SubscriptionDescription.DefaultMessageTimeToLive property to TimeSpan.FromDays(5).

When I create and send a brokered message, leaving the TimeToLive property on that message to the default value, when I inspect messages arriving in that the subscription, the ExpiresAtUtc property for the messages is 31-12-9999 23:59. and the TimetoLive property is 10675199.02:48:05.4775807. Since the TTL on the 'destination' subscription is 5 days, I would have expected a value 5.00:00:00 there.

Is there something else that I need to do to get this working?

Steps to reproduce with Service Bus Explorer 2.2.1.0

1. Connect to a Service Bus Namespace on azure
2. Create a new topic "MyTopic"
3. Create a subscription "Log" under the topic and set the Default Message Time to Live to 5 days.
4. Right click on the MyTopic topic node and choose "Send messages"
5. On the "Send messages to MyTopic" screen: click "Start"
6. Click the "Log" subscription node
7. Click the "Messages" button and choose Peek 10
8. Click the message in the list
9. Look at the TimeToLive field in the "Message Properties" area

Answer 1

Apparenlty the TTL value on peeked messages is meaningless:

This behavior is currently by design and should be seen independent of SDK version: the effective TTL is the minimum TTL of message and entity and is checked at runtime, rather than imprinted on the message at enqueue time (because the entity-level TTL could be changed at any time after messages were enqueued). The documentation is misleading and should be fixed.

This answer was given on this MSDN thread: http://social.msdn.microsoft.com/Forums/windowsazure/en-US/dfe58dbb-186d-4c71-a708-8f6f7267b451/when-peeking-for-messages-the-time-to-live-value-seems-not-to-be-set-is-this-a-bug

Answer 2

You have to set the DefaultMessageTimeToLive property when you create the queue itself and not when you create the queue client. Try deleting the queue and creating it again with the new queue description.

Answer 3

You need to log into your topic and set “Default Message Time to Live” to 14 days.

What went wrong? Luckily this error was self-explanatory and indicating that the Max Topic size is 1 GB (1073741824 bytes) while the Topic is already utilized 1073742326 bytes which translates to approx 1 GB limit. We logged into the Azure Portal and verified the same. Next we ran Service Bus Explorer to check the status of the messages. We saw thousands of messages are in the Topics.

Why messages were not getting cleared from the Topic? Now the obvious question arise, what is keeping those thousands of message in topic. After some investigation we found that “Default Message Time to Live” for Topic was set to very high number. In this case it was set to 1 year. In order to fix the issue, we set “Default Message Time to Live” to 2 days as recommended by the their App Developer. This resolved the issue. Here is a screen-shot of the configuration from Azure Management Portal:

See: https://blogs.msdn.microsoft.com/cie/2013/07/21/service-bus-error-the-maximum-entity-size-has-been-reached-or-exceeded-for-topic/