How to separate two bit maps of size 8 and 16 from a single PNG

Question

I have images from a SICK Trispector depth laser scanner. The image format is PNG. SICK calls it Trispector 2.5D PNG. The images contain both reflection data and depth data according to SICK's documentation. But SICK will not provide information on how to use this data without using their or partners' software. Essentially, what I need is the depth data. Reflection data might be a nice to have but is not necessary. The resulting image I get is monochrome. It seems to have the reflection data in the top part of the image and overlapping height data in the bottom. The scanned object is a crate of beer bottles with bottle caps. You can see an example here:

I have tried opening the image in many different image viewers and looked for information on 2.5D, but it does not appear to be relevant for this. In Matlab image preview I get one side of the height data individually, but I don't know how to use this information. See the following image from Matlab preview:

Does anyone know how to extrapolate the height data from an image like this? Maybe someone's worked with SICK's SOPAS or SICK scanners before, and understand this "2.5D PNG" format that SICK calls it. An OpenCV solution would be nice.

Edit: As @DanMašek comments, the problem is that of separating two images of different bitdepth from a single PNG. He provides further insight in the problem and a great OpenCV solution for separating the intensity and depth images as 8- and 16-bit, respectively:

Answer 1

Note: This information is based on the SICK TriSpector FAQ located on the SICK support forums (not publicly accessible, but you can request access).

---

The PNG images generated by SICK TriSpector store a concatenation of two pixel buffers:

• An 8-bit intensity image
• A 16-bit (little-endian) heightmap image

The resulting PNG image has the same width as each component, and thrice the height (since the PNG is 8-bit, and we have 3 bytes in total for each pixel position).

Let's consider a simple example, where the components have 3 rows and 4 columns. The data stored in the PNG will have the following structure:

The first step, as illustrated above, is to split the image into the two components. The PNG contains 9 rows, a third of that is 3 rows -- hence rows 0-2 contain the intensity, and the rest is the heightmap. The intensity image is directly usable, heightmap needs some further processing.

If we're on a little-endian architecture and are not concerned about portability, we can take advantage of the in-memory layout and just reinterpret the pixel buffer as 16 bit unsigned integers (in Python numpy.ndarray.view, in C++ create a new Mat wrapping the buffer).

The more flexible, albeit slower method is to combine the parts manually. Reshape the array to have the correct number of rows, then split it up based on odd or even column number (Skip indexing in Python). Convert each sub-array into 16bit unsigned integers, and finally combine them according to the formula LSB + 256 * HSB.

---

Example script in Python:

import cv2
import numpy as np

img = cv2.imread('combined.png', cv2.IMREAD_GRAYSCALE)
height, width = img.shape

# Determine the actual height of the component images
real_height = height/3

# Extract the top (intensity) part
intensity_img = img[:real_height,...]

# Extract the botton (heightmap) part
# Since this has two values per pixel, also reshape it to have the correct number of rows
heightmap_raw = img[real_height:,...].reshape(real_height,-1)

# The heightmap is 16 bit, two subsequent 8 bit pixels need to be combined
# ABCD -> A+256*B, C+256*D

# Quick but non-portable (needs little-endian architecture)
heightmap_img = heightmap_raw.view(np.uint16)

# Slower but portable
heightmap_a = heightmap_raw[...,::2].astype(np.uint16)
heightmap_b = heightmap_raw[...,1::2].astype(np.uint16)
heightmap_img = heightmap_a + 256 * heightmap_b

# Note: intensity is np.uint8, heightmap is np.uint16

cv2.imwrite('intensity.png', intensity_img)
cv2.imwrite('heightmap.png', heightmap_img)

---

Extracted intensity image:

Extracted height-map image (note that the original data was downscaled by factor of 256 while saving this for illustration, loosing a lot of the detail):