DataflowTemplates debezium connector issue

Question

I am referring this document to perform mysql(installed on local machine) to pubsub data streaming using debezium connector.

My properties file looks like below

databaseName=testdb 
databaseUsername=root 
databaseAddress=localhost 
databasePort=3306 
gcpProject=GCP_project_name 
databasePassword=password 
whitelistedTables=instance-name.testdb.testtab 
singleTopicMode=true 
gcpPubsubTopicPrefix=debeziumTest 
databaseManagementSystem=mysql

I have already created topic in pubsub with name "debeziumTest".

But the issue is, when i run sudo mvn exec:java -pl cdc-embedded-connector -Dexec.args="/path/to/properties-file"

, it runs without any error:

but there is no data uploaded to pubsub.

Answer 1

Based on the documentation, table updates are pushed to a topic that matches this format- ${PREFIX}${DB_INSTANCE}.${DATABASE}.${TABLE}

In your case I believe you should create a topic with the name - "debeziumTestinstance-name.testdb.testtab"

This may not be the only problem based on the warnings I see in the logs you shared.

Answer 2

The problem seems to be with your whitelistedTables.

According to the docuumentation, you should use ${instance}.${database}.${table}, for your given example it should be whitelistedTables=testdb.databaseName.testTab (if testTab is your tablename)