Load single channel 16-bit image to a texture in Unity3D

Question

I am working with a depth sensor (Orbbec Astra Pro) and want to display the infrared image in Unity. The data I receive is an ushort[] (or any other type, which is larger than 8bit).
So I can create a single channel 16-bit texture infraredTexture = new Texture2D(infraredFrame.Width, infraredFrame.Height, TextureFormat.R16, false);, but I do not know how to fill the texture with the infrared data, as LoadRawTextureData only takes a byte[] or IntPtr.

So in the end I would like to see a 16-bit grayscale texture in Unity. Is this even possible? Thank you in advance!

Answer 1

In general note from TextureFormat.R16

Currently, this texture format is only useful for runtime or native code plugins as there is no support for texture importing for this format.

Note that not all graphics cards support all texture formats, use SystemInfo.SupportsTextureFormat to check.

I don't have a direct solution for if you actually want to use a R16 format texture except somehow serialize the data to byte[] e.g. using Buffer.BlockCopy - NOTE: No idea of this would work at all!

ushort[] yourData = // wherever you get this from;
byte[] byteData = new byte[sizeof(ushort) * yourData.Length];

// On memory level copy the bytes from yourData into byteData
Buffer.BlockCopy(yourData, 0, byteData, 0, byteData.Length);

infraredTexture = new Texture2D(infraredFrame.Width, infraredFrame.Height, TextureFormat.R16, false);
infraredTexture.LoadRawTextureData(byteData);
infraredTexture.Apply();

Again no clue if it works that way.

---

However, I think you could simply "fake" it in Unity. If it is only for displaying the texture anyway you could rather use a "normal" RGB24 texture and simply map your 16-bit single channel data into a grey scale 24-bit RGB color and then apply it using Texture2D.SetPixels like e.g.

ushort[] yourData = // wherever you get this from;
var pixels = new Color[yourData.Length];

for(var i = 0; i < pixels.Length; i++)
{
    // Map the 16-bit ushort value (0 to 65535) into a normal 8-bit float value (0 to 1)
    var scale = (float)yourData[i] / ushort.MaxValue;
    // Then simply use that scale value for all three channels R,G and B
    // => grey scaled pixel colors
    var pixel = new Color(scale, scale, scale);
    pixels[i] = pixel;
}

infraredTexture = new Texture2D(infraredFrame.Width, infraredFrame.Height, TextureFormat.RGB24, false);

// Then rather use SetPixels to apply all these pixel colors to your texture
infraredTexture.SetPixels(pixels);
infraredTexture.Apply();