Adding audio to an incoming stream during video call to record voice of both parties in a call

Question

I have created an app using peer js to initiate video calls. I am using mediarecorder Api to record the incoming stream from caller. However, I need to add audio of both the caller and receiver in the call to the recording, and video should be of only the caller(incoming stream).

I have tried https://github.com/muaz-khan/MultiStreamsMixer this. However, on recording it I get an unreadable file by vlc.

I have also tried adding the local audio track to the recording stream, but that doesn't merge the 2 audio tracks into one and only the incomingstream's audio is recorded.

Answer 1

I was able to do this by using Web Audio API. I fetched the audio tracks from both the streams and joined them into one using audio context.

var OutgoingAudioMediaStream = new MediaStream();
OutgoingAudioMediaStream.addTrack(OutgoingStream.getAudioTracks()[0]);

var IncomingAudioMediaStream = new MediaStream();
IncomingAudioMediaStream.addTrack(IncomingStream.getAudioTracks()[0]);

const audioContext = new AudioContext();

audioIn_01 = audioContext.createMediaStreamSource(OutgoingAudioMediaStream);
audioIn_02 = audioContext.createMediaStreamSource(IncomingAudioMediaStream);

dest = audioContext.createMediaStreamDestination();

audioIn_01.connect(dest);
audioIn_02.connect(dest);

dest.stream.addTrack(IncomingStream.getVideoTracks()[0]);
var RecordingStream = dest.stream;

This worked perfectly.