Not able to create seekable video blobs from mediarecorder using EBML.js - MediaRecorder API - Chrome

Question

Using media recorder, I am able to upload and append the video blobs on azure. But combined video is not seekable on download with following code -

var chunks =[];
var mediaRecorder = new MediaRecorder(stream, 'video/x-matroska;codecs=vp8,opus');
mediaRecorder.ondataavailable = function(event) {
  if(event.data && event.data.size > 0) {
       chunks.push(event.data);
       appendBlockToAzure(chunks);
    }
};
mediaRecorder.start(10000);

I tried using EBML.js, if I use the following code then I get the seekable video file. This approach needs the file to be processed at the end. Therefore, final file could be of 1GB in size which will take very long time to upload.

var chunks =[];
var mediaRecorder = new MediaRecorder(stream, 'video/x-matroska;codecs=vp8,opus');
mediaRecorder.ondataavailable = function(event) {
   if(event.data && event.data.size > 0) {
        chunks.push(event.data);
        if(mediaRecorder.state == "inactive") { //if media recorder is stopped
            var combined = new Blob(chunks, { type: event.data.type });
            getSeekableBlob(combined, function (seekableBlob) {
                   saveCombinedVideoToAzure(seekableBlob);  
            });
        }
    }
 };
mediaRecorder.start(10000);

That's the reason I want to upload simultaneously to the azure. If I use the following code, then it logs unknown tag warnings and then length error. Also, the video file is not playable.

var seekablechunks =[];
var mediaRecorder = new MediaRecorder(stream, 'video/x-matroska;codecs=vp8,opus');
mediaRecorder.ondataavailable = function(event) {
   if(event.data && event.data.size > 0) {
        getSeekableBlob(event.data, function (seekableBlob) {
             seekablechunks.push(seekableBlob);
             saveCombinedVideoToAzure(seekablechunks);  
        });
   }
 };
mediaRecorder.start(10000);

Function 'getSeekableBlob':

function getSeekableBlob(inputBlob, callback) {
  // EBML.js copyrights goes to: https://github.com/legokichi/ts-ebml
  if(typeof EBML === 'undefined') {
      throw new Error('Please link: https://www.webrtc- experiment.com/EBML.js');
  }

  var reader = new EBML.Reader();
  var decoder = new EBML.Decoder();
  var tools = EBML.tools;

  var fileReader = new FileReader();
  fileReader.onload = function (e) {
      var ebmlElms = decoder.decode(this.result);
      ebmlElms.forEach(function (element) {
          reader.read(element);
      });
      reader.stop();
      var refinedMetadataBuf = tools.makeMetadataSeekable(reader.metadatas, reader.duration, reader.cues);
      var body = this.result.slice(reader.metadataSize);

      var newBlob = new Blob([refinedMetadataBuf, body], {
          type: 'video/webm'
      });

      callback(newBlob);
  };
  fileReader.readAsArrayBuffer(inputBlob);
}

Is there a way to get seekable blobs and upload them to azure?

Answer 1

It's a challenge for an open-ended streaming source for media (for example MediaRecorder) to create a file with SeekHead elements in it. The Seek elements in a SeekHead element contain byte offsets to elements in the file.

MediaRecorder doesn't create segments or SeekHead elements as you have discovered. To do so it would need to be able to see the future to know how big future compressed video and audio elements will be in the file.

A good way for you to handle this problem might be to post-process your uploaded files on a server. You can use ts-ebml to do this in a streaming fashion on a server when a file is completely uploaded.

It's possible, I suppose, to create Javascript software in your browser that can transform the stream of data emitted by MediaRecorder so it's seekable, on the fly. To make your stream seekeable you'd need to insert SeekHead elements every so often. You'd buffer up multiple seconds of the stream, then locate the Cluster elements in each buffer, then write a SeekHead element pointing to some of them. (Chrome's MediaRecorder outputs Clusters beginning with video key frames.) If you succeed in doing this you'll know a lot about Matroska / webm.

Answer 2

Suddenly, our Face on camera web-cam recorder component stopped saving webm blob.

In the console there were warnings about {EBML_ID: "55b0", type: "unknown", ...} during reader.read(element) and then "Uncaught (in promise) Error: No schema entry found for unknown" in EBMLEncoder.js" at tools.makeMetadataSeekable(...) call.

Ignoring unknown elements from the decoder workarounded the issue:

  ...
  var ebmlElms = decoder.decode(this.result);
  ebmlElms.forEach(function (element) {
      if (element.type !== 'unknown') {
          reader.read(element);
      }
  });
  reader.stop();
  ...

Related issue on ts-ebml npm package https://github.com/legokichi/ts-ebml/issues/33 with similar workaround