How to use deeplearn.js in a React component

148 views Asked by At

I am currently working on creating a project with react and deeplearn.js, and have reached a roadblock when combining the two. In my react application I am importing this deeplearnjs library model which I am using to do classification. Unfortunately, when I try to call the predict() method I get the following error:

TypeError: _this.variables is undefined

For the following part of code:

SqueezeNet.prototype.predictWithActivation = function (input, activationName) {
   var _this = this;
   var _a = this.math.scope(function () {
      var activation;
      var preprocessedInput = _this.math.subtract(input.asType('float32'), _this.preprocessOffset);

When I use the generated Javascript in a normal HTML it works perfectly, so I am unsure why I am getting this error within react. I have a feeling it has to do with stricter React rules or Javascript versioning, but I am not sure.

Thanks!

UPDATE

The simplest way to reproduce this is the following:

  1. Create a new React app with create-react-app
  2. Run yarn add deeplearn and yarn add deeplearn-squeezenet
  3. Modify App.js to the following:

    import React, { Component } from 'react';
    import {ENV, Array3D} from 'deeplearn';
    import {SqueezeNet} from 'deeplearn-squeezenet';
    
    class App extends Component {
      constructor() {
        super();
        var net = new SqueezeNet(ENV.math);
        net.load();
    
        var img = new Image(227, 227);
        img.src = 'boat.jpg';
        img.onload = function () {
          var pixels = Array3D.fromPixels(img)
          var res = net.predict(pixels);
        };
      }
      render() {
        return (
          <div></div>
        );
      }
    }
    
    export default App;
    
  4. Download the following file into the public folder: https://raw.githubusercontent.com/PAIR-code/deeplearnjs/master/models/squeezenet/cat.jpg

  5. Run yarn start

For reference I am using react 16.2.0

1

There are 1 answers

2
James Hibbard On BEST ANSWER

Your code is presumably failing because some of the method calls are asynchronous (.load() for example).

Here is how you would make your example work with React:

  1. Create a new React app with create-react-app
  2. Run yarn add deeplearn and yarn add deeplearn-squeezenet
  3. Add cat.jpg to the public folder
  4. Modify App.js as below

    import React, { Component } from 'react';
    import { ENV, Array3D } from 'deeplearn';
    import { SqueezeNet } from 'deeplearn-squeezenet';
    
    const math = ENV.math;
    const squeezeNet = new SqueezeNet(math);
    
    class App extends Component {
      constructor() {
        super();
        this.state = {
          statusText: 'Loading Squeezenet...'
        }
      }
    
      buildSuggestions(obj){
        return Object.keys(obj).map(
          key => `${obj[key].toFixed(5)}: ${key}`
        );
      }
    
      imageLoadHandler(e) {
        const img = e.target;
        squeezeNet.load()
          .then(() => {
            this.setState({ statusText: 'Predicting...' });
            const pixels = Array3D.fromPixels(img);
            const result = squeezeNet.predict(pixels);
            this.setState({ statusText: '' });
            squeezeNet.getTopKClasses(result, 5)
              .then((obj) => {
                this.setState({ statusText: this.buildSuggestions(obj) });
              });
          });
      }
    
      render() {
        const text = Array.isArray(this.state.statusText)?
          this.state.statusText :
          [this.state.statusText];
    
        return (
          <div>
            <img src="cat.jpg"
                 alt="cat"
                 onLoad={this.imageLoadHandler.bind(this)}
            />
            <div id="result">
              { text.map(el => <div key={el}>{el}</div>) }
            </div>
          </div>
        );
      }
    }
    
    export default App;
    
  5. Then run yarn start