Canvas to base64 Image without awt/swing classes

Question

I am trying for some time now to convert a Canvas into a base64 image String, so I am able to convert it back later to show on the screen or store it in a file. The problem is, all samples I found so far use SwingFXUtils.fromFXImage() or other AWT related classes.

The application I build will use the Gluon Substrate compiler to create mobile apps, which don't support all these classes. So is there a way to do this without any AWT/Swing related classes?

I get as far as this, resulting in an application/octet-stream, but I need an image mime type:

Canvas _canvas;
WritableImage image = new WritableImage((int) _canvas.getWidth(), (int) _canvas.getHeight());
_canvas.snapshot(new SnapshotParameters(), image);
int w = (int) image.getWidth();
int h = (int) image.getHeight();

byte[] buf = new byte[w * h * 4];
image.getPixelReader().getPixels(0, 0, w, h, PixelFormat.getByteBgraInstance(), buf, 0, w * 4);

String base64_image = McUtils.toBase64(buf);

Answer 1

You can encode the pixel data as a bitmap, then encode the bitmap data as a base64 encoded image data uri.

image/bmp Base64 encoding application

The bmp encoding algorithm was adapted from this algorithm by Philipp C. Heckel

BmpEncoderApp.java

import javafx.application.Application;
import javafx.scene.Group;
import javafx.scene.Scene;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;
import javafx.scene.paint.Color;
import javafx.scene.shape.Circle;
import javafx.stage.Stage;

import java.io.IOException;

public class BmpEncoderApp extends Application {
    public static void main(String[] args) {
        launch(args);
    }

    @Override
    public void start(Stage stage) throws IOException {
        // display a group to snapshot.
        Group root = new Group(
                new Circle(
                        10, 10, 10, Color.GREEN
                )
        );

        // snapshot the group.
        Scene scene = new Scene(root);
        Image snapshot = scene.snapshot(null);

        // display the snapshot.
        ImageView snapshotImageView = new ImageView(snapshot);
        snapshotImageView.setTranslateX(25);
        root.getChildren().add(snapshotImageView);

        // encode the snapshot as a base 64 encoded bitmap data URI.
        String dataUri = BmpUriEncoder.encode(snapshot);
        System.out.println(dataUri);

        // display the data uri in an image view.
        ImageView uriImageView = new ImageView(dataUri);
        uriImageView.setTranslateX(50);
        root.getChildren().add(uriImageView);

        stage.setScene(scene);
        stage.show();
    }
}

BmpUriEncoder.java

import javafx.scene.image.Image;
import javafx.scene.image.PixelFormat;

import java.io.*;
import java.nio.charset.StandardCharsets;
import java.util.Base64;

public class BmpUriEncoder {
    private static final int BMP_HEADER_LENGTH = 54;
    private static final int BMP_OFFSET_FILESIZE_BYTES = 2;
    private static final int BMP_OFFSET_IMAGE_WIDTH = 18;
    private static final int BMP_OFFSET_IMAGE_HEIGHT = 22;
    private static final int BMP_OFFSET_IMAGE_DATA_BYTES = 34;
    private static final int BMP_OFFSET_PAYLOAD_LENGTH = 38;
    private static final byte UDEF = 0x00;
    private static final byte[] BMP_HEADER = new byte[] {
            /* 00 */0x42, 0x4d, // signature
            /* 02 */UDEF, UDEF, UDEF, UDEF, // size in bytes, filled dynamically
            /* 06 */0x00, 0x00, // reserved, must be zero
            /* 08 */0x00, 0x00, // reserved, must be zero
            /* 10 */0x36, 0x00, 0x00, 0x00, // offset to start of image data in bytes
            /* 14 */0x28, 0x00, 0x00, 0x00, // size of BITMAPINFOHEADER structure, must be 40 (0x28)
            /* 18 */UDEF, UDEF, UDEF, UDEF, // image width in pixels, filled dynamically
            /* 22 */UDEF, UDEF, UDEF, UDEF, // image height in pixels, filled dynamically
            /* 26 */0x01, 0x00, // number of planes, must be 1
            /* 28 */0x20, 0x00, // number of bits per pixel (1, 4, 8 or 24) -> 24 = 0x18
            /* 30 */0x00, 0x00, 0x00, 0x00, // compression type (0=none, 1=RLE-8, 2=RLE-4)
            /* 34 */UDEF, UDEF, UDEF, UDEF, // size of image data in bytes (including padding)
            /* 38 */UDEF, UDEF, UDEF, UDEF, // normally: horizontal resolution in pixels per meter (unreliable)
            // HERE: used to indicate the payload length
            /* 42 */0x00, 0x00, 0x00, 0x00, // vertical resolution in pixels per meter (unreliable)
            /* 46 */0x00, 0x00, 0x00, 0x00, // number of colors in image, or zero
            /* 50 */0x00, 0x00, 0x00, 0x00, // number of important colors, or zero
    };

    private static byte[] encodeToBitmap(
            byte[] imageBytes,
            int w,
            int h
    ) throws IOException {
        int payloadLength = imageBytes.length;

        int filesizeBytes = w * h * 4 + BMP_HEADER_LENGTH; /* RGBA no line or file padding */

//        System.out.println("payload = " + payloadLength);
//        System.out.println("pixel width  = " + w);
//        System.out.println("pixel height = " + h);
//        System.out.println("filesize total = " + filesizeBytes);

        byte[] header = BMP_HEADER.clone();

        writeIntLE(header, BMP_OFFSET_FILESIZE_BYTES, filesizeBytes);
        writeIntLE(header, BMP_OFFSET_IMAGE_WIDTH, w);
        writeIntLE(header, BMP_OFFSET_IMAGE_HEIGHT, h);
        writeIntLE(header, BMP_OFFSET_IMAGE_DATA_BYTES, payloadLength);
        writeIntLE(header, BMP_OFFSET_PAYLOAD_LENGTH, payloadLength);

        ByteArrayOutputStream os = new ByteArrayOutputStream();
        os.write(header, 0, header.length);
        os.write(imageBytes);
        os.close();

        return os.toByteArray();
   }

    private static void writeIntLE(byte[] bytes, int startoffset, int value) {
        bytes[startoffset] = (byte) (value);
        bytes[startoffset + 1] = (byte) (value >>> 8);
        bytes[startoffset + 2] = (byte) (value >>> 16);
        bytes[startoffset + 3] = (byte) (value >>> 24);
    }

    public static String encode(Image image) throws IOException {
        int w = (int) image.getWidth();
        int h = (int) image.getHeight();
        int l = w * h * 4;

        byte[] pixelData = new byte[l];
        image.getPixelReader().getPixels(
                0, 0,
                w, h,
                PixelFormat.getByteBgraInstance(),
                pixelData,
                0,
                w * 4
        );

        // encode to a bitmap.
        byte[] bitmapData = encodeToBitmap(pixelData, w, h);

        // base 64 encode the output.
        byte[] base64Data = Base64.getEncoder().encode(
                bitmapData
        );

        // encapsulate base64 encoded data in a data uri.
        return
                "data:image/bmp;base64," +
                        new String(
                                base64Data,
                                StandardCharsets.US_ASCII
                        );
    }
}

Generated base 64 encoded image data URI

You can copy the resultant data uri and paste it into a browser navigation bar and the browser will display the snapshot image.

data:image/bmp;base64,Qk12BgAAAAAAADYAAAAoAAAAFAAAABQAAAABACAAAAAAAEAGAABABgAAAAAAAAAAAAAAAAAA////////////////////////////////nM6c/1CoUP8djh3/A4ID/wOCA/8djh3/UKhQ/5zOnP//////////////////////////////////////////////////////nM6c/x2OHf8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/x2OHf+czpz//////////////////////////////////////2m0af8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP9ptGn///////////////////////////9ptGn/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP9ptGn/////////////////nM6c/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP+czpz///////////8djh3/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/x2OHf//////nM6c/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/5zOnP9QqFD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/UKhQ/x2OHf8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8djh3/A4ID/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wOCA/8DggP/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/A4ID/x2OHf8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8djh3/UKhQ/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/1CoUP+czpz/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/nM6c//////8djh3/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/x2OHf///////////5zOnP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/nM6c/////////////////2m0af8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/2m0af///////////////////////////2m0af8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP9ptGn//////////////////////////////////////5zOnP8djh3/AIAA/wCAAP8AgAD/AIAA/wCAAP8AgAD/AIAA/wCAAP8djh3/nM6c//////////////////////////////////////////////////////+czpz/UKhQ/x2OHf8DggP/A4ID/x2OHf9QqFD/nM6c/////////////////////////////////w==

A couple of notes on this BMP implementation

1. It uses a slightly non-standard encoding of alpha data with the image data, which isn't that common for bitmaps. The solution could be adapted to not encode the alpha data into the bitmap with a bit of additional effort.
2. It doesn't encode the BMP meta data to actually be able to use the alpha data, so alpha data will not be displayed, it only represents opaque bitmaps.
3. You could configure the meta data so that the bitmap can display alpha data: By using a BI_BITFIELDS format as demoed in the wikipedia article on the BMP format. I tried that, but it didn't render correctly. I am not sure if that was due to an error in my encoding attempt or if support for such a format in standard renderers is limited.

If alpha channel support is required, I advise encoding to a png (for which I provide no solution) rather than using the bmp format, as png alpha channel support will be more reliable in potential client applications.

Encoding to other formats (png or jpg)

Usage of different formats that handle compression, such as png or jpg might be preferred for some tasks. For instance jpg is good for encoding photographic images in a small size and png is good at lossless compression with alpha channel support. But jpg and png are more complex to encode then a bmp format. If you wished to support the more advanced image formats, I suggest adapting an appropriate image encoder library to encode the data extracted from the JavaFX pixel writer.