Passing operators as functions to use with Pandas data frames

403 views Asked by At

I am selecting data from series on basis of threshold .

>>> s = pd.Series(np.random.randn(5))
>>> s
0   -0.308855
1   -0.031073
2    0.872700
3   -0.547615
4    0.633501
dtype: float64
>>> cfg = {'threshold' : 0 , 'op' : 'less' }
>>> ops = {'less' : '<', 'more': '>' , 'equal': '==' , 'not equal' : '!='}
>>> ops[cfg['op']]
'<'
>>> s[s < cfg['threshold']]
0   -0.308855
1   -0.031073
3   -0.547615
dtype: float64

I want to use ops[cfg['op']] in last line of code , instead of '<'. I am willing to change key , values of ops dict if required (like -lt instead of <). How this can be done?

2

There are 2 answers

1
piRSquared On BEST ANSWER

I'm all about @cᴏʟᴅsᴘᴇᴇᴅ's answer and @Zero's linked Q&A...
But here is an alternative with numexpr

import numexpr as ne

s[ne.evaluate('s {} {}'.format(ops[cfg['op']], cfg['threshold']))]

0   -0.308855
1   -0.031073
3   -0.547615
Name: A, dtype: float64

I reopened this question after having been closed as a dup of How to pass an operator to a python function?

The question and answers are great and I showed my appreciation with up votes.

Asking in the context of a pandas.Series opens it up to using answers that include numpy and numexpr. Whereas trying to answer the dup target with this answer would be pure nonsense.

3
this be Shiva On

Define a dictionary of methods that can stand in for your operators.

import operator    
d = {
         'more'  : operator.gt,
         'less'  : operator.lt,
         'equal' : operator.eq, 
         'not equal' : operator.ne
   }

Now, just index into your dictionary and apply your function parameters.

m = d[cfg['op']](s, cfg['threshold'])
m

0    False
1     True
2     True
3    False
4    False
dtype: bool

s[m]

1   -0.262054
2   -1.300810
dtype: float64

Here,

d[cfg['op']](s, cfg['threshold']) 

Is translated into

operator.lt(s, 0)