In Apache bookkeeper how do we delete a log entry in a ledger? If the ledge is immutable and cannot delete entries how does pulsar delete expired messages from bookies?
In Apache bookkeeper how do we delete a log entry in a ledger? If the ledge is immutable and cannot delete entries how does pulsar delete expired messages from bookies?
BookKeeper doesn't provide interfaces to delete individual entries in a ledger. It only provides methods to delete ledgers. Once the ledgers are deleted, bookies will garbage-collect their entries to reclaim disk space in background.
A pulsar topic partition is comprised of multiple ledgers. At any given of time, pulsar broker is writing to one ledger for a partition. After the ledger reaches a given size or after a certain period time, pulsar broker will close the ledger that it is writing and open a new ledger to write. pulsar keeps the list of ledgers as part of the metadata of a topic partition. If all the messages in a ledger have been consumed or expired, the ledger will be deleted by pulsar broker.
Following links are useful to help understand this:
[1] how a segment-based architecture delivers better performance, scalability, and resilience
[2] pulsar faq