Imagine a video call application where you allow the user to disable his camera stream (say, for privacy reasons) and in which it is also possible to take a picture with the camera to then send it to the peer via data channel. Now, if you want to do both at the same time (i.e. allow the user to have the video stream disabled to hide his flat, parts of his nudity or people around him, whatever, while handling the phone to take a picture that is not as critical in terms of privacy...), you're in a bit of trouble because when recording a picture with captureFrame
from the WebRTC api you need the stream to be enabled. What would a technical solution for this be?
First - and only - thing that sprang to my mind was enabling the camera stream and modifying it with insertable streams before sending it to the remote peer, such that the remote peer sees a black screen while the user can see his own camera stream and use it to record a photo. Would this be a viable solution?
I haven't tried this because I'm working with flutter_webrtc, which doesn't support insertable streams, and thus I'm looking for alternatives. Can you provide alternative solutions to this problem, ones that would work without insertable streams?
Found something which seems like a solution:
getUserMedia
, show it in a local renderer, use it for photo capture withcaptureFrame
, but do not add it to the peer connection.getUserMedia
and useRTCRtpSender.replaceTrack
to add video tracks to the peer connection (to avoid renegotiation/reconnects).Apart from some CameraException showing in logs on Android device this seems to be working.