How to test something rendered correctly?

Question

How does one test if an application rendered something correctly?

For example (2D example):

Microsoft Word 2007 http://img32.imageshack.us/img32/6197/37841144.png

How does one know that the shadow is placed correctly or the correct color / outline was rendered? Or if 3D effect renders correctly when one would rotate in a direction? Other things could be if the word art was re-sized, how does one measure its 'correctness'?

Answer 1

There are a few ways:

1. If it's actually very precisely specified what should be rendered and in exactly which way, then you can just compare the pixels to a reference rendering.
2. In case of things like SVG it's not so clearly specified. The usual approach here is to use reference renderings and compare them by hand. You can easily overlay both, subtract one from the other and spot glaring differences that way. It's not an automatic process, though.
3. You can look at the data representing the drawn image instead of the image on screen directly. What gets drawn is (in your example) a vector graphic. That means there are several shapes which should have well-defined properties, shapes and colors and you can simply compare your shape data to a reference. That kind of things can be done automatically. I think Google uses a similar approach to compare Chrome's rendering with reference renderings of web pages; they don't compare pixel data, they compare the higher-level data what and how the browser would render.

Answer 2

There are two ways: The image and the rendering based one.

The image way: You must find a way to render the image to an internal pixel buffer (so you can run your tests "headless", i.e. without an actual UI popping up).

Then select a few pixels and make sure their colors are correct. In your example, select a couple of white pixels around the text to make sure the rendering doesn't leak. Helper methods like assertThatRectangleIs(area, color) will help cover some ground. The idea is to select specific areas (like the left vertical bar of the H) and not be too picky.

The rendering based way works with the assumption that your gfx library works. So what you do is you mock the actual rendering code with something like this:

public class MockGC extends GC {
    List<Op> ops = new ArrayList<Op> ();
    void drawLine (int x1, int y1, int x2, int y2) {
        ops.add(new Line(x1, y1, x2, y2, copyGC (gc)));
    }
}

so you just save the command and all relevant options in a data structure that is easy to compare. Or even:

public class MockGC extends GC {
    StringBuilder buffer = new StringBuilder ();

    void drawLine (int x1, int y1, int x2, int y2) {
        buffer.append("line "+x1+","+y1+" - "+x2+","+y2+", color="+foreground()+"\n");
    }
}

Later, you can then just verify that the correct rendering commands have been issued and that the GC (colors, font, rendering hints) was used.

The latter way is much faster and 100% precise but much more work to code.