Cropping CGRect from AVCapturePhotoOutput (resizeAspectFill)

Question

I have found the following problem and unfortunatly other posts have not helped me to a working solution.

I have a simple app that shows the camera preview (AVCaptureVideoPreviewLayer) where the video gravity has been set to resizeAspectFill (videoGravity = .resizeAspectFill).

From my understanding this only streches the image in the width to make to fill the screen.

On my preview layer I also have applied a CGRect as a mask with fixed x, y, width and height.

Now once I take a photo i'm trying to crop that exact rectangle out of the image. For my understanding i'm supposed to use some kind of math to convert the CGRect to the same aspect ratio as the image that I get from the AVCapturePhotoOutput method but it never seems to crop correctly in the width.

    private func cropImage(image: UIImage) {
        let rect = CGRect(x: 25, y: 150, width: 325, height: 230)
        
        
        let scale = CGAffineTransform(scaleX: 1/self.view.frame.width, y: 1/self.view.frame.height)
        let flip = CGAffineTransform(scaleX: 1, y: -1).translatedBy(x: 0, y: -1)
        
        let bounds = rect.applying(scale).applying(flip)
        
        
        let topLeft = bounds.topLeft.scaled(to: image.size)
        let topRight = bounds.topRight.scaled(to: image.size)
        let bottomLeft = bounds.bottomLeft.scaled(to: image.size)
        let bottomRight = bounds.bottomRight.scaled(to: image.size)
        
        var ciImage = CIImage(image: image.forceSameOrientation())!
        
        ciImage = ciImage.applyingFilter("CIPerspectiveCorrection", parameters: [
            "inputTopLeft": CIVector(cgPoint: bottomLeft),
            "inputTopRight": CIVector(cgPoint: bottomRight),
            "inputBottomLeft": CIVector(cgPoint: topLeft),
            "inputBottomRight": CIVector(cgPoint: topRight)
        ])
        
        let context = CIContext()
        let cgImage = context.createCGImage(ciImage, from: ciImage.extent)
        let output = UIImage(cgImage: cgImage!)
        
        
        let vc = PreviewViewController()
        vc.imageView.image = output
        self.present(vc, animated: true, completion: nil)
    }

So again, basically it does crop at the correct height but its only the width that does not seem to go well.

Image example of what I would want to capture.

https://i.stack.imgur.com/zmeot.jpg

As you can see the bounding box in the top left stops after the "Q" button.

Result:

https://i.stack.imgur.com/z7RZd.jpg

As you can see in this image, it does crop correctly in the height however if we take a look at the top left it also includes half of the button to the left of the "Q" (Tab button)

Any help towards the solution would be appreciated!

Answer 1

I managed to solve the issue with this code.

private func cropToPreviewLayer(from originalImage: UIImage, toSizeOf rect: CGRect) -> UIImage? {
    guard let cgImage = originalImage.cgImage else { return nil }

    // This previewLayer is the AVCaptureVideoPreviewLayer which the resizeAspectFill and videoOrientation portrait has been set.
    let outputRect = previewLayer.metadataOutputRectConverted(fromLayerRect: rect)

    let width = CGFloat(cgImage.width)
    let height = CGFloat(cgImage.height)

    let cropRect = CGRect(x: (outputRect.origin.x * width), y: (outputRect.origin.y * height), width: (outputRect.size.width * width), height: (outputRect.size.height * height))

    if let croppedCGImage = cgImage.cropping(to: cropRect) {
        return UIImage(cgImage: croppedCGImage, scale: 1.0, orientation: originalImage.imageOrientation)
    }

    return nil
}

usage of the piece of code for my case:

let rect = CGRect(x: 25, y: 150, width: 325, height: 230)
let croppedImage = self.cropToPreviewLayer(from: image, toSizeOf: rect)
self.imageView.image = croppedImage