Is it possible to sum over multiple axis in numexpr?

1.7k views Asked by At

I am trying to do something like the following:

import numexpr as ne

a = np.random.rand(10, 1)
b = np.random.rand(1, 10)
ne.NumExpr('sum(sum(a*b, 1), 0)').run(a, b) # <- error: reduction operations must occur last
ne.NumExpr('sum(a*b, [1, 0])').run(a, b) # <- error: ValueError: cannot encode axis

This returns with the error documented here:

https://github.com/pydata/numexpr/wiki/Numexpr-Users-Guide#supported-reduction-operations

I was wondering if there is a workaround that I haven't thought of.

EDIT

To answer some comments:

This is a simpler example than the actual one I'm interested in. A more complete example exists here:

How to sum of squares of sum with memory limitations?

and here:

How to do a sum of sums of the square of sum of sums?

I don't expect you to read through those questions.

The primary reason I was interested in a numexpr implementation is that it offers easy support for multithreading compared to anything else I've seen, and the reduction operators reduce the need for memory storage which at times is crucial in my case.

I hope this clears a few things up.

2

There are 2 answers

1
wagnerpeer On

How about using two expressions?

import numpy as np
import numexpr as ne

a = np.random.rand(10, 1)
b = np.random.rand(1, 10)
ab = ne.evaluate('sum(a*b, 0)') # Calculates sum over first axis
print ne.evaluate('sum(ab, 0)') # <- Calculates sum over summed axis

I am not sure if this is a solution for you but this would avoid the limited usage of functions/sums in numexpr.

0
Bremsstrahlung On

You can sum over multiple axes using numexpr as follows:

import numpy as np
import numexpr as ne

a = np.random.rand(10, 1)
b = np.random.rand(1, 10)
ne.evaluate('sum(x, 0)', {'x': ne.evaluate('sum(a*b, 1)')})