Encoding yuv frames to video file in java

Question

I am trying to encode a video in java. I have access to the separate frames as I420 yuv frames (these come from a different part of the program that I cannot change). I basically have 3 bytebuffers for the different planes of a frame (+ dimensions). As far as I understand, my format has 1 byte for the y-plane, and half a byte for u and v each, per pixel.

What is the best way to encode these into an mp4 video file? I have tried with the xuggler API, but I can't seem to find a way to use the yuv frames directly. Right now, I would convert them to a BufferedImage (TYPE_3BYTE_BGR) first before I can use them with the xuggler api to encode them to a video.

But this creates a huge overhead (I have to convert the yuv data to rgb for each pixel) and is unnecessary, as xuggler encodes them to yuv frames again to store them in a video file? (Not sure about this.)

So is there any easier way to encode raw yuv-frames to a video file directly in java?

Thanks for any pointers.

Answer 1

The way your are planning this seems correct (manual conversion), as long as xuggler didn't re-encode the frame after you.

I have done this conversion with both python and C, the process is still the same as yours (frame by frame, looping the pixels). In Java, this could look like :

public class YUV2RGB
{

    public static void convert(int[] argb, byte[] yuv, int width, int height)
    {

        final int frameSize = width * height;

        final int ii = 0;
        final int ij = 0;
        final int di = +1;
        final int dj = +1;

        int a = 0;
        int y, v, u, r, g, b;

        for (int i = 0, ci = ii; i < height; ++i, ci += di)
        {
            for (int j = 0, cj = ij; j < width; ++j, cj += dj)
            {

                y = (0xff & ((int) yuv[ci * width + cj]));
                v = (0xff & ((int) yuv[frameSize + (ci >> 1) * width
                        + (cj & ~1) + 0]));
                u = (0xff & ((int) yuv[frameSize + (ci >> 1) * width
                        + (cj & ~1) + 1]));
                y = y < 16 ? 16 : y;

                // METHOD 1 [slower, less accurate]
                /*
                 * r = y + (int) 1.402f * v; g = y - (int) (0.344f * u + 0.714f
                 * * v); b = y + (int) 1.772f * u; r = r > 255 ? 255 : r < 0 ? 0
                 * : r; g = g > 255 ? 255 : g < 0 ? 0 : g; b = b > 255 ? 255 : b
                 * < 0 ? 0 : b; argb[a++] = 0xff000000 | (b<<16) | (g<<8) | r;
                 */

                // METHOD 2
                r = (int) (1.164f * (y - 16) + 1.596f * (v - 128));
                g = (int) (1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
                b = (int) (1.164f * (y - 16) + 2.018f * (u - 128));
                r = r < 0 ? 0 : (r > 255 ? 255 : r);
                g = g < 0 ? 0 : (g > 255 ? 255 : g);
                b = b < 0 ? 0 : (b > 255 ? 255 : b);
                argb[a++] = 0xff000000 | (r << 16) | (g << 8) | b;

            }
        }
    }

}

Sample code from https://github.com/jyanik/Mocobar/blob/master/Mocobar/src/com/yanik/mocobar/camera/YUV2RGB.java

Answer 2

I can´t remember the source of this code, probably it was from a question here in SO, but comes in handy here as it´s a version of the above using integer maths, I needed it for an Android project!

    //Method from Ketai project! Not mine! See below...
void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {

    final int frameSize = width * height;

    for (int j = 0, yp = 0; j < height; j++) {

        int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;

        for (int i = 0; i < width; i++, yp++) {

            int y = (0xff & ((int) yuv420sp[yp])) - 16;
            if (y < 0)
                y = 0;
            if ((i & 1) == 0) {
                v = (0xff & yuv420sp[uvp++]) - 128;
                u = (0xff & yuv420sp[uvp++]) - 128;
            }

            int y1192 = 1192 * y;
            int r = (y1192 + 1634 * v);
            int g = (y1192 - 833 * v - 400 * u);
            int b = (y1192 + 2066 * u);

            if (r < 0) r = 0;
            else if (r > 262143)
                r = 262143;
            if (g < 0) g = 0;
            else if (g > 262143)
                g = 262143;
            if (b < 0) b = 0;
            else if (b > 262143)
                b = 262143;

            rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
        }
    }
}