How to create an OrientedBoundingBox from Depth image and bounding box

Question

I want to crop a poincloud using the segmented mask (rectangle) and depth image, (I can consider that there is no rotation for now).

import open3d as o3d

## dummy pointcloud
demo_icp_pcds = o3d.data.DemoICPPointClouds()
pcd = o3d.io.read_point_cloud(demo_icp_pcds.paths[0])

## dummy depth mask
depth_mask = np.ones((640, 480), dtype=np.uint8)

## Crop PCD

## mask boundaries: [(xmin, xmax), (y_min, y_max)]
rect = [(20, 50), (70, 75)]

o_box = o3d.geometry.OrientedBoundingBox()
o_box.center = [0.0, 0.0, 0.0]
o_box.extent = [1.0, 2.0, 3.0]
o_box.R = [[1, 0, 0], [0, 1, 0], [0, 0, 1]]

cropped_pcd = pcd.crop(o_box)

I want to feed the info of the rect (x, y) and depth z to the orietned bounding box. Can anybody please tell me how can do that? thanks.

Answer 1

Cropping is much easier using axis aligned bounding box if your crop volume is cuboid and aligned with x,y,z axis. Anyways, an oriented bounding box with rotation as identity is equivalent. See below code for crop example.

import open3d as o3d

lower_b = [-2.0, -2.0, 1.0]
upper_b = [4.0, 2.0, 2.0]
bbox = o3d.geometry.AxisAlignedBoundingBox(min_bound=lower_b, max_bound=upper_b)
bbox.color = [0.46, 0.32, 1.0]
print(bbox, bbox.get_extent(), bbox.volume(), bbox.color)

ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.io.read_point_cloud(ply_point_cloud.path)

origin = o3d.geometry.TriangleMesh.create_coordinate_frame(size=1, origin=[0, 0, 0])
o3d.visualization.draw_geometries([origin, bbox, pcd],
                                  zoom=0.3412,
                                  front=[0.4257, -0.2125, -0.8795],
                                  lookat=[2.6172, 2.0475, 1.532],
                                  up=[-0.0694, -0.9768, 0.2024])
pcd1 = pcd.crop(bbox)
o3d.visualization.draw_geometries([origin, bbox, pcd1],
                                  zoom=0.3412,
                                  front=[0.4257, -0.2125, -0.8795],
                                  lookat=[2.6172, 2.0475, 1.532],
                                  up=[-0.0694, -0.9768, 0.2024])

obbox = o3d.geometry.OrientedBoundingBox(center=[2.0, 1.0, 1.5], extent=[1.0, 2.0, 3.0],
                                         R=[[1, 0, 0], [0, 1, 0], [0, 0, 1]])
pcd2 = pcd.crop(obbox)
obbox.color = [0.46, 0.32, 1.0]
o3d.visualization.draw_geometries([origin, obbox, pcd],
                                  zoom=0.3412,
                                  front=[0.4257, -0.2125, -0.8795],
                                  lookat=[2.6172, 2.0475, 1.532],
                                  up=[-0.0694, -0.9768, 0.2024])
o3d.visualization.draw_geometries([origin, obbox, pcd2],
                                  zoom=0.3412,
                                  front=[0.4257, -0.2125, -0.8795],
                                  lookat=[2.6172, 2.0475, 1.532],
                                  up=[-0.0694, -0.9768, 0.2024])

Also, follow 2 tips:-

• Add a coordinate axes for visualization of x,y,z axes. XYZ = RGB, i.e. red axis is X.
• Visualizer has white background and unfortunately, default color of bounding boxes is white. I am trying to get this changed, but the final decision will depend on repo maintainers.

Also, if you are using axis-aligned bounding box, be careful. Make sure min_bound < max_bound in all three axes. This has been fixed in recent PR by me - https://github.com/isl-org/Open3D/pull/6444, however changes are available only in development releases.