Find median for an image array for background estimation fast in Swift/ObjectiveC

Question

I need to estimate the background across multiple frames of a stationary video (not-moving camera). I have a number of frames and want to calculate the median for each pixel across all frames (usually 10 to 100 frames). I was able to do that with brute force but the performance is just awful (it takes 30-120 seconds to calculate). In Python with NumPy I can achieve this in a single np.median call:

medianFrames = [im1, im2, im3, im4]
medianFrame = np.median(medianFrames, axis=0).astype(dtype=np.uint8)   

In Objective-C the algorithm is below, very slow because it enumerates each pixel, creates an array for each component (R,G,B), and then calculates median value. It works but it's super slow (uses OpenCV Mat for image manipulation):

   for (int i = 0; i < result.rows; i++) {
        for (int j = 0; j < result.cols; j++) {
            NSMutableArray *elements_B = [NSMutableArray arrayWithCapacity:arr.count];
            NSMutableArray *elements_G = [NSMutableArray arrayWithCapacity:arr.count];
            NSMutableArray *elements_R = [NSMutableArray arrayWithCapacity:arr.count];
            for(int frameIndex = 0; frameIndex < arr.count; frameIndex++) {
                Mat frame = matArray[frameIndex];
                int B = frame.at<Vec3b>(i, j)[0];
                int G = frame.at<Vec3b>(i, j)[1];
                int R = frame.at<Vec3b>(i, j)[2];
                elements_B[frameIndex] = [NSNumber numberWithInt:B];
                elements_G[frameIndex] = [NSNumber numberWithInt:G];
                elements_R[frameIndex] = [NSNumber numberWithInt:R];
            }
            
            NSArray *sortedB = [elements_B sortedArrayUsingSelector:@selector(compare:)];
            NSUInteger middleB = [sortedB count] / 2;
            NSNumber *medianB = [sortedB objectAtIndex:middleB];
            
            result.at<Vec3b>(i,j)[0] = medianB.intValue;
            
            NSArray *sortedG = [elements_G sortedArrayUsingSelector:@selector(compare:)];
            NSUInteger middleG = [sortedG count] / 2;
            NSNumber *medianG = [sortedG objectAtIndex:middleG];
            
            result.at<Vec3b>(i,j)[1] = medianG.intValue;
            
            NSArray *sortedR = [elements_R sortedArrayUsingSelector:@selector(compare:)];
            NSUInteger middleR = [sortedR count] / 2;
            NSNumber *medianR = [sortedR objectAtIndex:middleR];
            
            result.at<Vec3b>(i,j)[2] = medianR.intValue;
        }
    }

The real bottleneck is an enumeration of each pixel across each image and calculating the median value. What is the best way to process multiple images and execute pixel-based math operations efficiently, as NumPy does?