pandas "cumulative" rolling_corr

Question

Is there any built-in pandas' method to find the cumulative correlation between two pandas series?

What it should do is effectively fixing the left side of the window in pandas.rolling_corr(data, window) so that the width of the window increases and eventually the window includes all data points.

Answer 1

Here's one way, map on index and apply corr for the increasing size of series.

In [116]: df.index.map(lambda x: df[col1].corr(df.loc[:x, col2]))

Details

In [112]: df = pd.DataFrame(pd.np.random.rand(10,2))

In [113]: df
Out[113]:
          0         1
0  0.094958  0.891910
1  0.482616  0.551912
2  0.877540  0.573768
3  0.839921  0.328452
4  0.334714  0.908346
5  0.530518  0.837590
6  0.285152  0.126937
7  0.386568  0.474815
8  0.279807  0.939694
9  0.741882  0.135982

In [114]: df['roll_corr'] = df.index.map(lambda x: df[0].corr(df.loc[:x, 1]))

In [115]: df
Out[115]:
          0         1  roll_corr
0  0.094958  0.891910        NaN
1  0.482616  0.551912  -1.000000
2  0.877540  0.573768  -0.832929
3  0.839921  0.328452  -0.848385
4  0.334714  0.908346  -0.839698
5  0.530518  0.837590  -0.791736
6  0.285152  0.126937  -0.312806
7  0.386568  0.474815  -0.283357
8  0.279807  0.939694  -0.354385
9  0.741882  0.135982  -0.459907

Validation

In [121]: df.corr()
Out[121]:
          0         1
0  1.000000 -0.459907
1 -0.459907  1.000000

In [122]: df[:5].corr()
Out[122]:
          0         1
0  1.000000 -0.839698
1 -0.839698  1.000000

Answer 2

Satyabrat Mishra is right. df.rolling(max_len, min_period = 1).func() could solve this question.

a = gen_df(100, 5)
b = gen_df(100, 5)
ans = a.rolling(100, min_periods = 10).corr(b)

Answer 3

Just use rolling correlation, with a very large window, and min_period = 1.