How to display text-labels in a open3d Point Cloud?

Question

I try to label certain points (obstacles) with text in my Point Cloud. Currently I have the following Point Cloud:

import open3d as o3d
import numpy as np

pcd = o3d.geometry.PointCloud()                                 # make new frame a pointcloud
pcd.points = o3d.utility.Vector3dVector(img)
obstacles = find_cluster(potential_obstacles)

o3d.visualization.draw_geometries([ pcd, obstacles ],
                                    zoom=0.41999999999999962,
                                    front=[ -0.49481467211023333, 0.062416027669252132, -0.86675410570382283 ],
                                    lookat= [ 0.20215540690279493, 0.25826321104010441, 2.5327259417027266 ],
                                    up=[ 0.068492446325020648, 0.99711550943685279, 0.032702379682518497 ],
                                    width=1920,
                                    height=1080)#

the function find_cluster() is not necessary for further understanding. It returns both the obstacles as colored point clouds numbered with labels and their position [x, y, z] as a numpy array.

I would like to use this information to display the "number" of the obstacle at its position in the point cloud as text.

point cloud with obstacle

I came across open3d.visualization.gui.Label3D but do not know how to use it. Can anyone help me out?

Thanks in advance and best regards, AV

I tried the folowing but did not succeed:

o3d.visualization.gui.Label3D('sample Text', [0, 0, 0])

How can I address this command to use the open window?

Answer 1

open3d.visualization.draw_geometries

You can use open3d.t.geometry.TriangleMesh.create_text to generate a 3D mesh of text and put it where you need it to be.

import open3d as o3d
from open3d.t.geometry import TriangleMesh

hello_open3d_mesh: TriangleMesh = o3d.t.geometry.TriangleMesh.create_text("Hello Open3D", depth=0.1).to_legacy()
hello_open3d_mesh.paint_uniform_color((0.4, 0.1, 0.9))


# Scale down since default mesh is quite big
# Location
location = (1, 3, 6)  # The mesh is not centered at origin, so there is already an offset.
# I am adding another location shift as an example.
hello_open3d_mesh.transform([[0.1, 0, 0, location[0]], [0, 0.1, 0, location[1]], [0, 0, 0.1, location[2]],
                             [0, 0, 0, 1]])
origin = o3d.geometry.TriangleMesh.create_coordinate_frame(size=0.3, origin=[0, 0, 0])

o3d.visualization.draw_geometries([origin, hello_open3d_mesh], mesh_show_back_face=True)

open3d.visualization.gui.Application

The o3d.visualization.gui.Label3D is used with open3d.visualization.gui.Application. See https://www.open3d.org/docs/release/python_api/open3d.visualization.gui.html and check its example here - https://www.open3d.org/docs/release/python_example/visualization/index.html.

open3d.visualization.O3DVisualizer

When using O3DVisualizer, it has add_3d_label method which allows to do the same. See docs.