Use GridSearchCV in a pipeline with a nested estimator

Question

I try to use a Pipeline to build my model like this: I want to predict multiple outputs with a random forst classifier. Since a pipeline only allows only the last step to be the classifier, I nested my pipeline. This works fine without GridSearch.

pipeline = Pipeline([
('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', MultiOutputClassifier(RandomForestClassifier(), n_jobs=-1)),
])

Now I try to pass multiple params to my RF classifier, but since it is nested, it will be passed to the MultiOutputClassifier, at least that´s what I think happens.

param_grid = { 
    'clf__n_estimators': [200, 500],
    'clf__max_features': ['auto', 'sqrt', 'log2'],
    'clf__max_depth' : [4,5,6,7,8],
    'clf__criterion' :['gini', 'entropy']
}

cv = GridSearchCV(pipeline, param_grid=param_grid)

This results in an error: ValueError: Invalid parameter criterion for estimator

Is there a way to pass the params to my RandomForestClassifier or is there a way to pipe multiple classifiers?

Answer 1

Try this:

pipeline = Pipeline([
('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', MultiOutputClassifier(RandomForestClassifier(), n_jobs=-1)),
])

param_grid = { 
    'clf__estimator__n_estimators': [200, 500],
    'clf__estimator__max_features': ['auto', 'sqrt', 'log2'],
    'clf__estimator__max_depth' : [4,5,6,7,8],
    'clf__estimator__criterion' :['gini', 'entropy']
}

cv = GridSearchCV(pipeline, param_grid=param_grid, n_jobs=2)

In general, you can access tunable params with:

cv.get_params().keys()
dict_keys(['cv', 'error_score', 'estimator__memory', 'estimator__steps', 'estimator__verbose', 'estimator__vect', 'estimator__tfidf', 'estimator__clf', 'estimator__vect__analyzer', 'estimator__vect__binary', 'estimator__vect__decode_error', 'estimator__vect__dtype', 'estimator__vect__encoding', 'estimator__vect__input', 'estimator__vect__lowercase', 'estimator__vect__max_df', 'estimator__vect__max_features', 'estimator__vect__min_df', 'estimator__vect__ngram_range', 'estimator__vect__preprocessor', 'estimator__vect__stop_words', 'estimator__vect__strip_accents', 'estimator__vect__token_pattern', 'estimator__vect__tokenizer', 'estimator__vect__vocabulary', 'estimator__tfidf__norm', 'estimator__tfidf__smooth_idf', 'estimator__tfidf__sublinear_tf', 'estimator__tfidf__use_idf', 'estimator__clf__estimator__bootstrap', 'estimator__clf__estimator__ccp_alpha', 'estimator__clf__estimator__class_weight', 'estimator__clf__estimator__criterion', 'estimator__clf__estimator__max_depth', 'estimator__clf__estimator__max_features', 'estimator__clf__estimator__max_leaf_nodes', 'estimator__clf__estimator__max_samples', 'estimator__clf__estimator__min_impurity_decrease', 'estimator__clf__estimator__min_impurity_split', 'estimator__clf__estimator__min_samples_leaf', 'estimator__clf__estimator__min_samples_split', 'estimator__clf__estimator__min_weight_fraction_leaf', 'estimator__clf__estimator__n_estimators', 'estimator__clf__estimator__n_jobs', 'estimator__clf__estimator__oob_score', 'estimator__clf__estimator__random_state', 'estimator__clf__estimator__verbose', 'estimator__clf__estimator__warm_start', 'estimator__clf__estimator', 'estimator__clf__n_jobs', 'estimator', 'iid', 'n_jobs', 'param_grid', 'pre_dispatch', 'refit', 'return_train_score', 'scoring', 'verbose'])