I have a python script that reads messages from a Kafka topic, run some custom filtering, and produce a new message to another Kafka topic.
Currently the script accepts 2 arguments:
--target_topic. script pseudo code:
for each message in source_topic: is_fit = check_if_message_fits_target_topic(message) if is_fit: produce(target_topic, message)
and i run my script like:
python3 my_script.py --source_topic someSourceTopic --target_topic someTargetTopic
My wish is to be able to make the function
check_if_message_fits_target_topic to be dynamic so i can run the same script on-demand with different custom parameters.
argparse to manage the topic names arguments. What is the best way to pass a whole function as argument?
Just for the sake of the example, i have a running app that apply:
def check_if_message_fits_target_topic(message): values = message.value if values['event_name'] == 'some_event_name': return True return False
I want to build it in a generic way so i will be able to push some other custom logic, for example:
def check_if_message_fits_target_topic(message): values = message.value yesterday = datetime.date.today() - datetime.timedelta(days=1) if values['created_at'] > yesterday: return True return False
check_if_message_fits_target_topic should be able to do anything i pass it, as long it returns True or False.