Passing parameter Name in confluent HDFS Connector

Question

Is there any way we can pass the topic names, file name location from config file in the connect HDFS properties. Right now , we are passing topic name like this :

Topic = <topic_1> , <topic_2>
File location = <File_directory>

If we are getting topic names in the config file then we need to find the way to pass topic name like this :

Topic=cat <config_file> | grep topic_name | cut 1-7 ( This is unix way to read config file.. I am sure we must have some way to read config file in the connect HDFS properties )

Answer 1

Kafka Connect reads the Configuration as a Map object in key value pairs.

The topics and topics.dir determine where on HDFS the files are written to.

If you want to parse the actual config file locally, you can use grep or jq (assuming you are using Distributed connect)