C# Design guidance for generic image processing library

Question

In any image processing library there is always the issue of having to provide an implementation of every single algorithm for every single image format (color space, channels, bit depth, mem layout, etc). One very elegant solution to the problem is Boost GIL. Through the power of C++, and a great design, all of these issues are abstracted away and you can write a single algorithm that will work on any type of image.

I wanted to create something similar in C#, but many of the necessary constructs such as templates and certain operator overloads (like unary *) are missing. I'm willing to accept that what I can create will not be nearly as robust and elegant as GIL, but to the extent possible I would like to model the concepts. Ultimately, abstracting image differences and writing generic processing algorithms would be the aim.

With what there is available in C#, generics, lambdas, even dynamic IL /cringe, what do people think some possible approaches would be to designing the library?

Answer 1

Have you seen Aforge.NET, the way that's designed is pretty generic.

The author of that library solved a lot of the problems you are talking about through interfaces. Off the top of my head stuff like, IFilter, IFilterColourToAny etc

To do efficient image processing in .NET you'll need to go into unsafe code using Bitmap.LockData (I think), which could negate all the cool .NET stuff you're talking about...

Answer 2

Though my answer comes awfully late, I hope it will be useful for other people.

Given the limitations of C# that the OP has identified so far, here is a list of criteria that would still give limited freedom to the programmer in terms of pixel formats.

• A flexible bitmap handle C# class, which supports a variety of pixel formats without being a C# generic class
  • Because the generic class in the C# language would not have afforded any benefit to the user of image processing.
• Some safety guarantee - certain characteristics of a bitmap handle object should be immutable
  • In particular, the bitmap's pixel format and pixel dimensions should not be allowed to change once created. To change these, a new object must be created.
• Provides both mutable interface (allows change to pixel values) and immutable object model interface
  • To provide an indication on any API function that there should be no attempt to write to a particular bitmap argument.
• A set of image processing algorithm classes that accept and return the bitmap handle class.
  • Each algorithm tries its best to handle different kinds of pixel formats, to the extent that is economically feasible.

---

With the criteria in mind, I would recommend using System.Windows.Media.Imaging as the substrate of the library you are building.

The System.Windows.Media.Imaging namespace is the C# counterpart to the Microsoft Windows Imaging Component (WIC) library. Therefore, the underlying processing is implemented in native C++, giving it the speed needed for practical use.

Thanks to the wide range of pixel format support implemented in WIC, the C# counterpart also supports the same range of pixel formats.

---

The WIC (and System.Windows.Media.Imaging) does not provide any advanced image processing capability (nothing of the sort of Canny edge detection, Hough transform, object detection, etc.)

However, in terms of being an in-memory bitmap object interchange interface (for integrating different image libraries with C# interfaces or bindings), both System.Windows.Media.Imaging.WriteableBitmap and System.Drawing.Bitmap are suitable.

---

In terms of implementing algorithms, it is sometimes difficult to make algorithms equally applicable to single-channel images and multi-channel images. Such requires years if not decades of research in multivariate mathematics.

Thus, it is common for image processing algorithm classes to focus on supporting a narrow subset of pixel formats:

• 1bpp Black White (for logical / decision bitmaps, e.g. edge map or connected component blob membership)
• 8bpp Gray
• 24bpp BGR
• 32bpp BGRA
• 32bpp Gray Float
• 96bpp BGR Float
• 128bpp BGRA Float

If an algorithm class sees an input bitmap handle that is not one of the above type, it would try its best to losslessly "promote" the input format into one of the above formats.

With this automatic up-conversion in mind, the user of this algorithm class loses strict control on the output bitmap's pixel format, but gains the guarantee that the output's visual appearance will be in accordance with expectation.