Features in vowpal wabbit

Question

I need to select a combination (quadratic and cubic) of features from subset of all namespaces. I want to avoid features which are single. Example:

1 1 '1-1-48112-c|m m_18 |o o_15 |s s_100390 |t t_20 |n n_5283 |d d_10023 |p p_2606 |w w_1 |f f_6.0 |g g_113119

I want to select all features (as per command line of VW I use option -q m: ) which are combination of with namespace 'm'. Importantly, I want to avoid all single feature like t_20, m_18, o_15, etc.

using --ignore params would also remove the combinations, which I don't want.

In the source code it seems ignore namespaces are removed very early before the combinations are made. I am willing to change the source code of VW if needed.

Answer 1

If you want to use the interactions between 2 namespaces, while simultaneously ignoring the 2 namespaces on their own, you are going to need to construct the interaction features on your own and pass them to the model.

It shouldn't be too hard: just contruct a new namespace with the concatenation of features you want, e.g: 1 1 '1-1-48112-c|mynewnamespace t_20_m_18_o_15

However, I'm curious why you want to do this? Including the base features plus their quadratic interactions will probably yield similar, if not better, predictive accuracy as taking out the base features.