How can I draw rectangular paths at specified times in swift using CAKeyframeAnimation?

Question

I am writing an app where I take videos and run each frame through a cormel object detection model. With the output of this model, I would like to draw the bounding boxes around each detected object for each frame in the video. To do this, I believe I should use CAKeyFrameAnimation. My attempt is below:

    private func addBoundingBox(to layer: CALayer, videoSize: CGSize, coordinates: [[[Float]]], confidences: [[[Float]]], timeStamps: [Double]){
        let shapeLayer = CAShapeLayer()
        shapeLayer.frame = CGRect(origin: .zero, size: videoSize)
        
        let pathAnimation = CAKeyframeAnimation(keyPath: "path")
        var paths = [CGPath]()
        
        for frame in 0..<coordinates.count{
            var path = CGMutablePath()
            for i in 0..<coordinates[frame].count {
                var box = CGRect(x: Double(coordinates[frame][i][0]), y: 1 - Double(coordinates[frame][i][1]), width: Double(coordinates[frame][i][2]), height: Double(coordinates[frame][i][3]))
                
                let rectPath = CGPath(rect: LocalToShiftedPixelRect(local: box, videoSize: videoSize), transform: nil)
                path.addPath(rectPath)
            }
            paths.append(path)
        }
        
        pathAnimation.values = paths
        pathAnimation.keyTimes = timeStamps as [NSNumber]
        pathAnimation.beginTime = AVCoreAnimationBeginTimeAtZero
        
        shapeLayer.add(pathAnimation, forKey: "path")
        
        //shapeLayer.path = path
        shapeLayer.strokeColor = UIColor.red.cgColor
        shapeLayer.fillColor = UIColor.clear.cgColor
        shapeLayer.lineWidth = 3;
        
        layer.addSublayer(shapeLayer)
    }
    
    private func localToPixel(local: CGRect, videoSize: CGSize) -> CGRect{
        var pixelRect = CGRect(x: local.midX*videoSize.width, y: local.midY*videoSize.height, width: local.width*videoSize.width, height: local.height*videoSize.height)
        
        return pixelRect
    }
    
    private func ShiftForRectangleCenter(local: CGRect) -> CGRect{
        var shiftedRect = CGRect(x: local.minX - local.width/2, y: local.minY - local.height, width: local.width, height: local.height)
        
        return shiftedRect
    }
    
    private func LocalToShiftedPixelRect(local: CGRect, videoSize: CGSize) -> CGRect{
        return localToPixel(local: ShiftForRectangleCenter(local: local), videoSize: videoSize)
    }

However, when I run this, I see approximately the first half of my pathAnimation values show up on the screen, but the path animation is going way faster than the video, even though the key values correspond with the time stamps for each frame. For some reason I do not see any of the paths drawn for what should correspond to the second half of the animation.

Also, to whoever has knowledge on this, would you mind sharing how you managed to learn the animation system? The apple documentation does not seem to have much at all as far as useful examples for this animation stuff, so most of what I've tried has come from a mixed bag of tutorials and stack overflow answers.

Thanks!