Python - Pass a function as argument/parameter to create a dynamic script

Question

I have a python script that reads messages from a Kafka topic, run some custom filtering, and produce a new message to another Kafka topic.

Currently the script accepts 2 arguments: --source_topic and --target_topic. script pseudo code:

for each message in source_topic:
    is_fit = check_if_message_fits_target_topic(message)
    if is_fit:
        produce(target_topic, message)

and i run my script like: python3 my_script.py --source_topic someSourceTopic --target_topic someTargetTopic

---

My wish is to be able to make the function check_if_message_fits_target_topic to be dynamic so i can run the same script on-demand with different custom parameters.

I'm using argparse to manage the topic names arguments. What is the best way to pass a whole function as argument?

Just for the sake of the example, i have a running app that apply:

def check_if_message_fits_target_topic(message):
    values = message.value
    if values['event_name'] == 'some_event_name':
        return True
    return False

I want to build it in a generic way so i will be able to push some other custom logic, for example:

def check_if_message_fits_target_topic(message):
    values = message.value
    yesterday = datetime.date.today() - datetime.timedelta(days=1)

    if values['created_at'] > yesterday:
        return True
    return False

check_if_message_fits_target_topic should be able to do anything i pass it, as long it returns True or False.

Answer 1

Here is a simple example that sets a default implementation of a method that can be overridden by passing the name of a python module on the command line. I would personally do something more nuanced with classes I think, but this might give you one path forward.

main.py

import sys
import importlib

def fn():
    print("default function implementation")

if len(sys.argv) == 2:
    fn = importlib.import_module(sys.argv[1]).fn

fn()

custom.py

def fn():
    print("custom function implementation")

Now you can do:

python main.py

and get

default function implementation

or

python main.py custom

and get

custom function implementation