TypeError: Cannot read property 'classify' of undefined (save ml5.js model )

Question

I am using an ml5.js setup in my React app where I click on a button to train a model and then I click on another button to make predictions. The Test button works on the first time but it throws an error on the second time:

TypeError: Cannot read property 'classify' of undefined

I believe this is because once I run the prediction part, the model gets deleted? Because the classifier is now undefined. How can I modify it such that I can repeatedly click on the Test button and obtain new predictions each time.

I tried the save() function but apparently it only downloads the model and not save it for the app.

export const Video: React.FC<ComponentProps> = (props: ComponentProps) => {
    const [prediction, setPrediction] = useState<string>();

    let capture: p5Types.Element;
    let classifier: any;
    const setup = (p5: p5Types, canvasParentRef: Element) => {
        capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
        const featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
        classifier = featureExtractor.classification(capture, videoReady);
    }

    const draw = (p5: p5Types) => {
    }
    function gotResult() {
        classifier.classify(capture, (err: any, result: any) => {
            setPrediction(result[0].label);
        });
    }

    function train() {
        classifier.train((lossValue: any) => {
            console.log('Loss is', lossValue);
        });
        //classifier.save();
    }



    return (<div><Sketch setup={setup} draw={draw} className="sketch" />
        <div className="button">
            <Button variant="contained" color="primary" onClick={() => classifier.addImage('first')}>First</Button>
            <Button variant="contained" color="primary" onClick={() => classifier.addImage('second')}>Second</Button>
        </div>
        <div className="secondbutton">
            <Button variant="contained" color="primary" onClick={() => train()}>Train!</Button>
            <Button variant="contained" color="primary" onClick={() => gotResult()}>Test!</Button>
            <br />
            <span>Prediction: {prediction}</span>
        </div>
    </div>)
        ;
};

Codesandbox:

https://codesandbox.io/s/hardcore-solomon-zb34l?file=/src/Component.tsx

Updated code:

export const VideoComponent: React.FC<ComponentProps> = (props: ComponentProps) => {
    const [prediction, setPrediction] = useState<string>();
    const [confidence, setConfidence] = useState<string>();
    const [trainingComplete, setTrainingComplete] = useState<boolean>();
    const captureRef = useRef<p5Types.Element>();
const classifierRef = useRef<any>();

    const setup = (p5: p5Types, canvasParentRef: Element) => {

        const capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
    const featureExtractor = ml5.featureExtractor("MobileNet", modelReady);
    captureRef.current = capture;
    classifierRef.current = featureExtractor.classification(capture, videoReady);
    }

    const draw = (p5: p5Types) => {
    }


    function gotResult() {
        console.log('classifier in results', classifierRef.current);
        classifierRef.current.classify(captureRef.current, (err: any, result: any) => {
            setPrediction(result[0].label);
            setConfidence(result[0].confidence);

        });
    }

    function train() {
        console.log('classifier in train', classifierRef.current);
        classifierRef.current?.classify.train((lossValue: any) => {
            console.log('Loss is', lossValue);
            if (lossValue == null) {
                //setTrainingComplete(true);
                console.log('training complete')
            }
        });
    }



    return (
    <div>
        <Sketch setup={setup} draw={draw} className="sketch" />
        <div className="button">
            <Button variant="contained" color="primary" onClick={() => { classifierRef.current?.classifier.addImage('first'); console.log('image added') }}>First</Button>
            <Button variant="contained" color="primary" onClick={() => { classifierRef.current?.classifier.addImage('second'); console.log('image added') }}>Second</Button>
        </div>
        <div className="secondbutton">
            <Button variant="contained" color="primary" onClick={() => train()}>Train!</Button>
            <Button variant="contained" color="primary" onClick={() => gotResult()}>Test!</Button>
            <br />
            {trainingComplete && (<span>Training Complete!</span>)}<br />
            <span>Prediction: {prediction}</span><br />
        </div>
    </div>)
        ;
};

Answer 1

The problem is the two let variables, capture and classifier, which you are using to store stateful data.

let capture: p5Types.Element;
let classifier: any;
const setup = (p5: p5Types, canvasParentRef: Element) => {
    capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
    const featureExtractor = ml5.featureExtractor('MobileNet', modelReady);
    classifier = featureExtractor.classification(capture, videoReady);
}

These variables get re-created on every re-render. If you want to have instance variables which persist across re-renders then you need to use useState or useRef. Since these two variables come from the p5 package and might be modified by other functions I think useRef is probably what you want here. You can read more about useRef in the React docs.

You need to add some extra conditionally checks in your code since captureRef.current could be either an Element or undefined, so you need to make sure that you have an Element before using it. For classifier you can use the optional chaining operator like this: classifierRef.current?.classify().

const captureRef = useRef<p5Types.Element>();
const classifierRef = useRef<any>();

const setup = (p5: p5Types, canvasParentRef: Element) => {
    // I'm assigning to a variable before assigning to the ref
    // so that we know that it is always defined inside this function
    const capture = p5.createCapture(p5.VIDEO).parent(canvasParentRef);
    const featureExtractor = ml5.featureExtractor("MobileNet", modelReady);
    captureRef.current = capture;
    classifierRef.current = featureExtractor.classification(capture, videoReady);
    console.log("start", classifierRef.current);
};

You should try to find or create an actual type for classifier instead of any. It looks like there is no types package for ml5, but someone created a draft.