iOS Vision Framework image rectification

1.1k views Asked by At

I'd like to perform rectification of a pair of images using the iOS Vision Framework's VNHomographicImageRegistrationRequest. Is it possible?

So far, I've obtained a 3x3 warp matrix that doesn't seem to rectify the images.

How is the warp matrix supposed to be used (I couldn't find any examples online)?

Moreover, how does image alignment differ from image rectification (I understand image rectification, but not image alignment)?

Xcode Playground:

import UIKit
import Foundation
import Vision

var li = UIImage(named: "left.png")
var ri = UIImage(named: "right.png")

let handler = VNSequenceRequestHandler()
let request = VNHomographicImageRegistrationRequest(targetedCGImage: li!.cgImage!, options: [:]) { (req, err) in
    let observation = req.results?.first as! VNImageHomographicAlignmentObservation
    print(observation.warpTransform)
}

try! handler.perform([request], on: ri!.cgImage!)

OpenCV image warping:

import numpy as np
import cv2

# read the pair of images
li = cv2.imread('left.png', 0)
ri = cv2.imread('right.png', 0)

# get the 3x3 warp matrix provided by Vision Framework
ios_vision_warp_mat = np.transpose(np.array([
    [0.746783, -0.0139349, -0.000149109],
    [-0.0426033, 0.861793, -2.39433e-05],
    [133.91, 22.0962, 0.999471]
]))

# warp the image
warped = cv2.warpPerspective(ri, ios_vision_warp_mat, (ri.shape[1], ri.shape[0]))
combined = cv2.addWeighted(warped, 0.5, li, 0.5, 0.0)

cv2.imshow('Combined pair', combined)
cv2.imshow('Unrectified pair', np.concatenate([li, warped], axis=1))

cv2.waitKey(0)
0

There are 0 answers