executor.map and named parameter passing

Question

I have a case of a function which must be run with a large number of parameter types

# generic function with named parameters
def genericFn(X, y, a_fn ='t', o_fn ='a', l_fn='b', dim=2):
   ...
  return accuracy

I have a list of parameter types

a_fns = ['t', 's', 'r', 'l']
o_fns = ['s', 'a']
l_fns = ['m', 'h', 'l', 'b', 'k', 'p']

How can I pass every possible combination of the functions to the pool

with concurrent.futures.ProcessPoolExecutor() as executor:
  results_list = executor.map(genericFn, (combinations of named parameters))

Answer 1

Remember that you need to adapt examples to your needs. You could use something like this:

from functools import partial

class dict_to_kwargs:
    def __init__(self, func):
        self.func = func

    def __call__(self, arg, **kwargs):
        return self.func(**kwargs, **arg)

class your_class:
    def your_method(self):
        result_executor = executor.map(
                    dict_to_kwargs(
                        partial(
                            method_name_to_map,
                            named_param_1=named_unvariable_param_1_value,
                            named_param_2=named_unvariable_param_2_value,
                            named_param_3=named_unvariable_param_3_value,
                            named_param_4=named_unvariable_param_4_value,
                            named_param_5=named_unvariable_param_5_value,
                        )
                    ),
                    [{"named_param_6": x} for x in named_variable_param_6_values],
                )
        return list(result_executor) # to return it like a list

• 'method_name_to_map' is the method you want to send params to and iterate.
• 'named_unvariable_param' is for values that should be the same on each iteration.
• 'named_variable_param' if for you to pass params that are variable and should be iterated.
• 'dict_to_kwargs class' is one of the tricks here. It is used to convert what you pass, as kwargs.
• 'partial method' per documentation:

"Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords."