Good day!
I was using pgsync for change data capture (CDC) from postgres (wal=logical) to elasticsearch. What I noticed while using pgsync is that it captures all data from postgres on initial run.
Because of circumstances (high customization), now I need to use custom CDC from postgres (and, possibly, mongodb thru oplog) to elasticsearch. Now what I can do is streaming and capturing data via wal2json.
However, I cannot get to replicate all data from postgres. Logical slots allow fetching data from the moment of creation, not before.
What can be done?
PS. Simple approach would be selecting all data from table chuck by chunk on initial creation. Is it appropriate to do it? If so, how can I convert data from select statements to format as in 'pg_logical_slot_get_changes' result?