Creating MediaStream from canvas and video element

3.7k views Asked by At

I am creating a MediaStream object and adding a video track to it from a canvas using the captureStream() function. This works fine.

However I am trying to add audio as a separate track from a video element. I cant seem to find a way to get an AudioTrack object from a html video element.

Currently HTMLMediaElement.audioTracks is not supported in Chrome. According to the mozilla developer site I should be able to use HTMLMediaElement.captureStream() to return a MediaStream object from which I should be able to retrieve the separate tracks but I just get 'captureStream is not a function' error.

Perhaps i'm missing something very obvious but I would greatly appreciate any help on this.

Below is my current code:

var stream = new MediaStream();

//Works fine for adding video source    
var videotracks = myCanvas.captureStream().getTracks();
var videostream = videotracks[0];
stream.addTrack(videostream);

//Currently not supported in Chrome
var audiotracks = myVid.audioTracks;
var audiostream = audiotracks[0];
stream.addTrack(audiostream);
1

There are 1 answers

0
Kaiido On BEST ANSWER

To get an audio stream from a video element in a cross-browser way :

AudioContext API createMediaStreamDestination + createMediaElementSource.

// if all you need is the audio, then you should even probably load your video in an Audio element
var vid = document.createElement('video');
vid.onloadedmetadata = generateAudioStream;
vid.crossOrigin = 'anonymous';
vid.src = 'https://dl.dropboxusercontent.com/s/bch2j17v6ny4ako/movie720p.mp4';

function generateAudioStream() {
  var audioCtx = new AudioContext();
  // create a stream from our AudioContext
  var dest = audioCtx.createMediaStreamDestination();
  // connect our video element's output to the stream
  var sourceNode = audioCtx.createMediaElementSource(this);
  sourceNode.connect(dest)
    // start the video
  this.play();
  // your audio stream  
  doSomethingWith(dest.stream)
}

function doSomethingWith(audioStream) {
  // the audio element that will be shown in the doc
  var output = new Audio();
  output.srcObject = audioStream;
  output.controls = true;
  output.play();
  document.body.appendChild(output);
}

To add audio to a canvas stream :

MediaStream Capture Canvas and Audio Simultaneously