Unable to establish WebRTC connection with Node JS server as a peer

Question

I am trying to send images captured from a canvas to my NodeJS backend server using the WebRTC data channel. That is I am trying to make my server a peer. But for some reason, I am unable to establish a connection.

Client Side

async function initChannel()
{
    const offer = await peer.createOffer();
    await peer.setLocalDescription(offer);

    const response = await fetch("/connect", {
        headers: {
            'Content-Type': 'application/json',
        },
        method: 'post',
        body: JSON.stringify({ sdp: offer, id: Math.random() })
    }).then((res) => res.json());

    peer.setRemoteDescription(response.sdp);

    const imageChannel = peer.createDataChannel("imageChannel", { ordered: false, maxPacketLifeTime: 100 });

    peer.addEventListener("icecandidate", console.log);
    peer.addEventListener("icegatheringstatechange",console.log);

     // drawCanvas function draws images got from the server.
    imageChannel.addEventListener("message", message => drawCanvas(remoteCanvasCtx, message.data, imageChannel));
                                   
     // captureImage function captures and sends image to server using imageChannel.send()
    imageChannel.addEventListener("open", () => captureImage(recordCanvasCtx, recordCanvas, imageChannel));
}

const peer = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.stunprotocol.org:3478" }] });
initChannel();

Here both captureImage and drawCanvas are not being invoked.

Server Side

import webrtc from "wrtc"; // The wrtc module ( npm i wrtc )

function handleChannel(channel)
{
    console.log(channel.label); // This function is not being called. 
}

app.use(express.static(resolve(__dirname, "public")))
    .use(bodyParser.json())
    .use(bodyParser.urlencoded({ extended: true }));

app.post("/connect", async ({ body }, res) =>
{
    console.log("Connecting to client...");

    let answer, id = body.id;

    const peer = new webrtc.RTCPeerConnection({ iceServers: [{ urls: "stun:stun.stunprotocol.org:3478" }] });
    await peer.setRemoteDescription(new webrtc.RTCSessionDescription(body.sdp));
    await peer.setLocalDescription(answer = await peer.createAnswer());

    peer.addEventListener("datachannel",handleChannel)

    return res.json({ sdp: answer });
});

app.listen(process.env.PORT || 2000);

Here the post request is handled fine but handleChannel is never called.

---

When I run this I don't get any errors but when I check the connection status it shows "new" forever. I console logged remote and local description and they seem to be all set. What am I doing wrong here?

I am pretty new to WebRTC and I am not even sure if this is the correct approach to continuously send images (frames of user's webcam feed) to and back from the server, if anyone can tell me a better way please do.

And one more thing, how can I send image blobs ( got from canvas.toBlob() ) via the data channel with low latency.

Answer 1

I finally figured this out with the help of a friend of mine. The problem was that I have to create DataChannel before calling peer.createOffer(). peer.onnegotiationneeded callback is only called once the a channel is created. Usually this happens when you create a media channel ( either audio or video ) by passing a stream to WebRTC, but here since I am not using them I have to to this this way.

Client Side

const peer = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.l.google.com:19302" }] });
const imageChannel = peer.createDataChannel("imageChannel");

imageChannel.onmessage = ({ data }) => 
{
    // Do something with received data.
};

imageChannel.onopen = () => imageChannel.send(imageData);// Data channel opened, start sending data.

peer.onnegotiationneeded = initChannel

async function initChannel()
{
    const offer = await peer.createOffer();
    await peer.setLocalDescription(offer);

    // Send offer and fetch answer from the server
    const { sdp } = await fetch("/connect", { 
        headers: {
            "Content-Type": "application/json",
        },
        method: "post",
        body: JSON.stringify({ sdp: peer.localDescription }),
    })
        .then(res => res.json());

    peer.setRemoteDescription(new RTCSessionDescription(sdp));
}

Server

Receive offer from client sent via post request. Create an answer for it and send as response.

app.post('/connect', async ({ body }, res) =>
{
    const peer = new webrtc.RTCPeerConnection({
        iceServers: [{ urls: 'stun:stun.l.google.com:19302' }],
    });
    console.log('Connecting to client...');
    peer.ondatachannel = handleChannel;

    await peer.setRemoteDescription(new webrtc.RTCSessionDescription(body.sdp));
    await peer.setLocalDescription(await peer.createAnswer());

    return res.json({ sdp: peer.localDescription });
});

The function to handle data channel.

/**
 * This function is called once a data channel is ready.
 *
 * @param {{ type: 'datachannel', channel: RTCDataChannel }} event
 */
function handleChannel({ channel })
{
    channel.addEventListener("message", {data} =>
    {
        // Do something with data received from client. 
    });
    
    // Can use the channel to send data to client.
    channel.send("Hi from server");
}

So here is what happens :

1. Client creates a Data-Channel.
2. Once data channel is created onnegotiationneeded callback is called.
3. The client creates an offer and sends it to the server (as post request).
4. Server receives the offer and creates an answer.
5. Server sends the answer back to the client (as post response).
6. Client completes the initialization using the received answer.
7. ondatachannel callback gets called on the server and the client.

I have used post request here to exchange offer and answer but it should be fairly easy to do the same using Web Socket if that is what you prefer.