I'd like to perform rectification of a pair of images using the iOS Vision Framework's VNHomographicImageRegistrationRequest
. Is it possible?
So far, I've obtained a 3x3 warp matrix that doesn't seem to rectify the images.
How is the warp matrix supposed to be used (I couldn't find any examples online)?
Moreover, how does image alignment differ from image rectification (I understand image rectification, but not image alignment)?
Xcode Playground:
import UIKit
import Foundation
import Vision
var li = UIImage(named: "left.png")
var ri = UIImage(named: "right.png")
let handler = VNSequenceRequestHandler()
let request = VNHomographicImageRegistrationRequest(targetedCGImage: li!.cgImage!, options: [:]) { (req, err) in
let observation = req.results?.first as! VNImageHomographicAlignmentObservation
print(observation.warpTransform)
}
try! handler.perform([request], on: ri!.cgImage!)
OpenCV image warping:
import numpy as np
import cv2
# read the pair of images
li = cv2.imread('left.png', 0)
ri = cv2.imread('right.png', 0)
# get the 3x3 warp matrix provided by Vision Framework
ios_vision_warp_mat = np.transpose(np.array([
[0.746783, -0.0139349, -0.000149109],
[-0.0426033, 0.861793, -2.39433e-05],
[133.91, 22.0962, 0.999471]
]))
# warp the image
warped = cv2.warpPerspective(ri, ios_vision_warp_mat, (ri.shape[1], ri.shape[0]))
combined = cv2.addWeighted(warped, 0.5, li, 0.5, 0.0)
cv2.imshow('Combined pair', combined)
cv2.imshow('Unrectified pair', np.concatenate([li, warped], axis=1))
cv2.waitKey(0)