WebRTC: Capture picture while stream to remote peer disabled

221 views Asked by At

Imagine a video call application where you allow the user to disable his camera stream (say, for privacy reasons) and in which it is also possible to take a picture with the camera to then send it to the peer via data channel. Now, if you want to do both at the same time (i.e. allow the user to have the video stream disabled to hide his flat, parts of his nudity or people around him, whatever, while handling the phone to take a picture that is not as critical in terms of privacy...), you're in a bit of trouble because when recording a picture with captureFrame from the WebRTC api you need the stream to be enabled. What would a technical solution for this be?

First - and only - thing that sprang to my mind was enabling the camera stream and modifying it with insertable streams before sending it to the remote peer, such that the remote peer sees a black screen while the user can see his own camera stream and use it to record a photo. Would this be a viable solution?

I haven't tried this because I'm working with flutter_webrtc, which doesn't support insertable streams, and thus I'm looking for alternatives. Can you provide alternative solutions to this problem, ones that would work without insertable streams?

1

There are 1 answers

0
jakobleck On

Found something which seems like a solution:

  1. Stop video track of the (already disabled) local stream that was added to the peer connection.
  2. Create new local stream with getUserMedia, show it in a local renderer, use it for photo capture with captureFrame, but do not add it to the peer connection.
  3. After photo capture, stop this newly created stream.
  4. Restart local stream with getUserMedia and use RTCRtpSender.replaceTrack to add video tracks to the peer connection (to avoid renegotiation/reconnects).

Apart from some CameraException showing in logs on Android device this seems to be working.