Create tensor with a single ROI from cv::Mat and cv::Rect with OpenVINO API 2.0

Question

OpenVINO API 2.0 changed how images are preprocessed.

Documentation describes an overload used to construct a region of interest tensor from another tensor.

Tensor(const Tensor& other, const Coordinate& begin, const Coordinate& end)

I found a brief example but did not find it clear enough to translate my use case.

/** input_tensor points to input of a previous network and
    cropROI contains coordinates of output bounding box **/
ov::Tensor input_tensor(ov::element::f32, ov::Shape({1, 3, 20, 20}));
ov::Coordinate begin({0, 0, 0, 0});
ov::Coordinate end({1, 2, 3, 3});
//...

My goal is to take an image (cv::Mat) and bounding box (cv::Rect) and execute inference on the ROI without having to copy the ROI memory before calling set_input_tensor (replacement for SetBlob). I have not been able to find sufficient examples or documentation for how to achieve this. I am currently unsure how to translate a cv::Rect into ov::Coordinates.

Previously I was able to do this with API 1.0 using the following example:

const InferenceEngine::ROI roi(0, bounding_box.x, bounding_box.y, bounding_box.width, bounding_box.height);
const InferenceEngine::TensorDesc tensor_desc(InferenceEngine::Precision::U8, { 1, channels, image_height, image_width }, InferenceEngine::Layout::NHWC);
const InferenceEngine::TensorDesc roi_tensor_desc = InferenceEngine::make_roi_desc(tensor_desc, roi, true);
const InferenceEngine::Blob::Ptr image_blob = InferenceEngine::make_shared_blob<uint8_t>(roi_tensor_desc, image.data);
request.SetBlob(input_name, image_blob);

Answer 1

OpenVINO API 2.0 changed layout work for models with NHWC. If your model has input { 1, channels, image_height, image_width } with NHWC layout, after conversion to IR with cmd '--layout=input_1(NHWC)' '--input_shape=[1, image_height, image_width, channels]', you have the input shape [1, image_height, image_width, channels]. Thus, you need to make ROI relative to this size.

For example, to make ROI tensor, you need:

const auto cv::Rect bounding_box = <>;
const auto shared_tensor = ov::Tensor(<data>); // it's tensor with shared input data
// N, H, W, C
const auto begin = ov::Coordinate({0, bounding_box.y, bounding_box.x, 0}); // the coordinates first bounding box corner
const auto end = ov::Coordinate({1, bounding_box.y + bounding_box.height, bounding_box.x + bounding_box.width, 3}); // the coordinates second bounding box corner
const auto roi_tensor = ov::Tensor(shared_tensor, begin, end); 

ov::Coordinate is class to represent coordinates on shapes. In this case these coordinates are of the bounding box.

Answer 2

Previously in legacy API, you need to specify h and w dimensions.

Meanwhile, in OV 2.0 API you need to specify the starting and ending of coordinates for the ROI tensor.

You may refer to here for more info.