twilio-video-processors.js - React project problem

Question

I am facing an issue when trying to implement twilio-video-processors.js on my react app.

My component attached below.

import { Box, Stack } from "@mui/material";
import * as React from "react";
import { useEffect } from "react";
import { useRef } from "react";
import Twilio from "twilio-video";
import vsnBG from "../Resources/vsn_bg.png";
import { useTheme } from "@emotion/react";
import * as VideoProcessors from "@twilio/video-processors";


const Test = () => {
  const videoRef = useRef(null);
  const theme = useTheme();

  useEffect(() => {
    
    const showLocalVideo = async () => {
      const videoTrack = await Twilio.createLocalVideoTrack({
        width: 640,
        height: 480,
        frameRate: 24,
      });

      videoRef.current.appendChild(videoTrack.attach());

      let img = new Image();
      img.src = vsnBG;
      img.onload = async () => {
        const bg = new VideoProcessors.VirtualBackgroundProcessor({
          assetsPath: "/",
          backgroundImage: img,
          maskBlurRadius: 5,
        });

        await bg.loadModel();

        videoTrack.addProcessor(bg);
      };
    };
    showLocalVideo();
  }, []);

  return (
    <div>
      <div id="video" ref={videoRef}></div>
    </div>
  );
};

export default Test;

These are errors I am getting while running app app: I tried many solutions but nothing works...

ERROR
Unexpected token '<'
SyntaxError: Unexpected token '<'
ERROR
Unexpected token '<'
SyntaxError: Unexpected token '<'
ERROR
window.createTwilioTFLiteModule is not a function
TypeError: window.createTwilioTFLiteModule is not a function
    at VirtualBackgroundProcessor.<anonymous> (http://localhost:3000/main.b5cd24c700b1f035cd03.hot-update.js:730:41)
    at step (http://localhost:3000/main.b5cd24c700b1f035cd03.hot-update.js:392:17)
    at Object.next (http://localhost:3000/main.b5cd24c700b1f035cd03.hot-update.js:341:14)
    at fulfilled (http://localhost:3000/main.b5cd24c700b1f035cd03.hot-update.js:300:24)

I tried debugging and going thoght the documention. I also tried to run it only on node.js without react with vanilla JS and it worked fine, but still not working on react.

Answer 1

I've reviewed the error message you provided, and it appears that the issue can be resolved by following the steps below:

Add the following script to your package.json file:

"postinstall": "rimraf public && copyfiles -f node_modules/@twilio/video-processors/dist/build/* dist"

This script ensures that the necessary files are copied to the /dist directory, which is where Parcel serves your application from.

Verify that the files have been successfully copied to the /dist directory. In your code, set the assetsPath property to '/'. This informs the video processor where to locate the model files.

Here's an example of how to do this in a React application:

import * as Video from 'twilio-video'
import * as VideoProcessors from '@twilio/video-processors'

const videoTrack = await Video.createLocalVideoTrack({
  width: props.width,
  height: props.height,
  frameRate: 24
});

const bg = new VideoProcessors.GaussianBlurBackgroundProcessor({
  assetsPath: '/',
  maskBlurRadius: 10,
  blurFilterRadius: 5,
});

await bg.loadModel();

videoTrack.addProcessor(bg);

Once you've completed these steps, you should be able to use the blur or virtual background features as described in the documentation.