I have 2 time-series of matrices that I would like to multiply together. I have the data stored as a pandas MultiIndex frame. Both frames share the same first axis, which are k dates. So, the first matrix A has k dates, and for each date it has a n x n matrix, making it k x n x n. The second matrix B also has k dates and for each date it has an n by m matrix, making it k x n x m. For each date k, I would like to multiple the n x n part of A with the n x m part of B. I have changed my MultiIndex into a numpy array in a bid to speed up the operation as k is circa 15,000,000.
I have set this up as: AB = np.array([np.dot(A.loc[date_], B.loc[date_]) for date_ in B.index.levels[0]]). Unfortunately, this takes many, many hours to run. I have heard that using np.einsum is much, much quicker, but I have thusfar been unable to get it to work. Does anyone have any suggestions on how to speed this up and if possible, use np.einsum to achieve that same result in less time?
Python: multiply 2 time series of matrices
54 views Asked by The User At
1
There are 1 answers
Related Questions in PANDAS
- ModuleNotFoundError on .ipynb
- Str object is not callable in pandas
- Need help realigning python fill_between with data points
- AttributeError: module 'numba' has no attribute 'generated_jit'
- Fix error when assigning a list of values to dataframe row
- How to make pandas show large datasets in output?
- merge dataframe but do not sort by merge key
- vim python omnifunc not working some modules
- Preserving DataFrame Modifications Across Options in a Streamlit Application
- How to join 2 datasets by looking up based on a string (full match or part match)
- Python Pandas getting hierarchy path till top management
- How to convert pandas series to integer for use in datetime.fromisocalendar
- reformat numbers stored in array
- How can I resolve this error and work smoothly in deep learning?
- What is the best way to merge two dataframes that one of them has date ranges and the other one has date WITHOUT any shared columns?
Related Questions in NUMPY
- Why numpy.vectorize calls vectorized function more times than elements in the vector?
- Producing filtered random samples which can be replicated using the same seed
- Numpy array methods are faster than numpy functions?
- When I create a series of spectrograms from a long audio file, the colour intesities vary noticably
- How do I fix a NumPy ValueError for an inhomogeneous array shape?
- How should I troubleshoot "RuntimeWarning: invalid value encountered in arccos" in NumPy?
- Unravel by multi-index/group
- Calculating IRR Using Numpy
- Integrating with an array of upper limits without sacrificing time efficiency
- Why doesn't this code work? - Backpropagation algorithm
- How to remove integers from a mixed numpy array containing sub-arrays and integers?
- How to transfer object dataframe in sklearn.ensemble methods
- Rust cannot borrow as mutable
- Why does the following code detect this matrix as a non-singular matrix?
- How to detect the exact boundary of a Sudoku using OpenCV when there are multiple external boundaries?
Related Questions in MATRIX-MULTIPLICATION
- Using the sympy module to compute the matrix multiplication involving symbols
- How can glPushMatrix affect the rotation of an object around a rotating object?
- Handling Memory Insufficiency in Graph Convolutional Network (GCN) for Large Graphs
- Multiply vector embedding column with itself to generate similarity scores for all combinations in spark dataframe
- Why do these two approaches to calculating the MSE gradient in R not give the same result?
- Feeding a Transformer with a matrix
- in cuda kernel , the shared memory matrix As is transposed, resulting in an error
- CUDA float matrix multiplication gives the wrong answer
- Are camera calibration matrices (intrinsic (K) and extrinsic (P)) supposed to be unique or is only the homography KP unique
- Correlation matrix shrinkage causes matrix multiplication error for monte carlo simulation
- Facing error in Strassen's Matrix Multiplication Algorithm
- Multiply two matrices column-wise to obtain vector
- How to matrix multiply each column from two matrices in numpy?
- How to multiply two integer square matrices using MSVC inline assembly in C++
- Why does this inverse matrix encryption, decryption not work with more than 1 digit matrix as a key?
Related Questions in NUMPY-EINSUM
- numpy dot product of sub array?
- Multiplying elementwise over final axis of two arrays
- removing loops with numpy.einsum
- Inner product of Tensors
- Einsum in python
- replacing einsum with normal operations
- Loss of accuracy when computing a product between 2 tensors?
- Understand the details of einsum application for two tensors
- Use numpy.einsum to calculate the covariance matrix of data
- Dot-product a list of Matrices in numpy
- How to improve this bottleneck calculation in Python (use of C++?)
- Trying to implement 2d image convolution with numpy throws ValueError: operands could not be broadcast together
- Speeding up einsum for specific matrix and vector size
- einsum equivalent for ndarray multiplication
- Matrix Vector Product across Multiple Dimensions
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
After playing around with einsum for hours, I finally managed to solve this:
np.einsum('ijk,ikl->ijl', A, B)