Vnc with WebRTC for remote screen control

Question

Overview

Is it possible to use VNC (RFB) with WebRTC to implement remote screen control using Node Js? I get remote screen frames from RFB and I want to transform it to MediaStream and then send to the client side. I was trying to search for any solution in the net but found nothing I can use.

Possible solutions I've found

1. ffmpeg frame encoding (I don't sure I can encode frames to something suitable for MediaStream)
2. put frames into canvas element and then capture to the MediaStream

Main question

How to encode rfb frames to be suitable for Mediastream and WebRTC

What I've been using until now

I just transform rfb frames to png pictures, send to the client and render it using canvas. Problem - poor fps, quite big latency

Is there any other solutions except WebRTC?

Answer 1

I think WebRTC is a great solution for this, this Open Source project neko does it already. They aren't using VNC (but instead using GStreamer to capture X11), but that totally possible to change.

Since png is lossless you are wasting a lot of bandwidth on that, if possible I would encode to VPx or H264.

Are you transporting these png via the DataChannel? I would also use RTP if possible. The browser will discard late frames (and other optimizations) to make sure you get the best experience.