How to play WEBM files individually which are created by MediaRecorder

Question

For recording audio and video, I am creating webm files under the ondataavailable of MediaRecorder API. I have to play each created webm file individually.

Mediarecorder api inserts header information into first chunk (webm file) only, so rest of the chunks do not play individually without the header information.

As suggested link 1 and link 2, I have extracted the header information from first chunk,

// for the most regular webm files, the header information exists
// between 0 to 189 Uint8 array elements

const headerIinformation = arrayBufferFirstChunk.slice(0, 189); 

and perpended this header information into second chunk, still the second chunk could not play, but this time the browser is showing poster (single frame) of video and duration of sum of two chunks, eg:10 seconds; duration of each chunk is 5 second.

The same header-information thing I have done with the hex editor. I opened the webm file in editor and copied the first 190 elements from first webm file and put this into second file, something like below image, even this time, the second webm file could not play and the result was same as in previous example.

Red color is showing the header information:

This time I copied the header and cluster information from first webm file placed this into second file, something like below image, but did not get success,

Questions

What I am doing wrong here ?

Is there any way that we can play the webm files/chunks individually ?

Note: I can't use the MediaSource to play those chunks.

Edit 1

As @Brad suggested, I want to insert all the content before the first cluster to a later a cluster. I have few webm files that each has duration of 5 seconds. After digging into the files, I came to know, almost every alternate file hasn't cluster point (no 0x1F43B675).

Here I am confused that I'll have to insert header information (initialization data) at the beginning of every file or beginning of every first cluster? If I choose a later option, then how's going to play the webm file that doesn't have any cluster ?

Or, first I need to make each webm file in a way that it has cluster at very beginning, so I can prepend the header information before cluster in those files?

Edit 2

After some digging and reading this , I came up with the conculsion that each webm file needs header info, cluster and actual data.

Answer 1

// for the most regular webm files, the header information exists

// between 0 to 189 Uint8 array elements

Without seeing the actual file data it's hard to say, but this is possibly wrong. The "header information" needs to be everything up to the first Cluster element. That is, you want to keep all data from the start of the file up to before you see 0x1F43B675 and treat it as initialization data. This can/will vary from file to file. In my test file, this occurs a little after 1 KB in.

and perpended this header information into second chunk, still the second chunk could not play, but this time the browser is showing poster (single frame) of video and duration of sum of two chunks, eg:10 seconds; duration of each chunk is 5 second.

The chunks output from the MediaRecorder aren't relevant for segmentation, and can occur at various times. You would actually want to split on the Cluster element. That means you need to parse this WebM file, at least to the point of splitting out Clusters when their identifier 0x1F43B675 comes by.

Is there any way that we can play the webm files/chunks individually ?

You're on the right path, just prepend everything before the first Cluster to a later Cluster.

Once you've got that working, the next problem you'll likely hit is that you won't be able to do this with just any cluster. The first Cluster must begin with a keyframe or the browser won't decode it. Chrome will skip over to the next cluster, to a point, but it isn't reliable. Unfortunately, there's no way to configure keyframe placement with MediaRecorder. If you're lucky enough to be able to process this video server-side, here's how to do it with FFmpeg: https://stackoverflow.com/a/45172617/362536

Answer 2

Okay looks like this is not as easy as you have to scan through the blob to find the magic value.

let offset = -1;
let value = 0;
const magicNumber = parseInt("0x1F43B675".match(/[a-fA-F0-9]{2}/g).reverse().join(''), 16)

while(value !== magicNumber) {
  offset = offset + 1;

  try {
    const arr = await firstChunk.slice(offset, offset + 4).arrayBuffer().then(buffer => new Int32Array(buffer));
    value = arr[0];
  }
  catch(error) {
    return;
  }
}

offset = offset + 4;

The answer is 193 199

const header = firstChunk.slice(0, offset);
const blobType = firstChunk.type;
const blob = new Blob([header, chunk], { type: blobType });

And there you have it. Now question is how did I get this number? Why is it not multiple of 42?

Brute force

Well the logic is simple, record the video, gather chunks, slice the first chunk, compute new blob and try to play it with HTMLVideoElement. If it fails increase the offset.

(async() => {

    const microphoneAudioStream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });

    const mediaRecorder = new MediaRecorder(microphoneAudioStream);
    let chunks = [];

    mediaRecorder.addEventListener('dataavailable', (event) => {
        const blob = event.data;
        chunks = [...chunks, blob];
    });

    mediaRecorder.addEventListener("stop", async () => {
        const [firstChunk, ...restofChunks] = chunks;
        const [secondBlob] = restofChunks;
        const blobType = firstChunk.type;

        let index = 0;
        const video = document.createElement("video");

        while(index < 1000) {
            const header = firstChunk.slice(0, index);
            const blob = new Blob([header, secondBlob], { type: blobType });
            const url = window.URL.createObjectURL(blob);

            try {
                video.setAttribute("src", url);
                await video.play();
                console.log(index);
                break;
            }
            catch(error) {

            }

            window.URL.revokeObjectURL(url);

            index++;
        }
    })

    mediaRecorder.start(200);

    const stop = () => {
        mediaRecorder.stop();
    }

    setTimeout(stop, 400)

})();

I noticed that for smaller timeslice param in MediaRecorder.start and timeout param in setTimeout the header offset becomes 1. Sadly still not 42.