DeepNetts 1.3 is having issues with the serialization on the setEarlyStopping and writeToFile any network with ADAM optimizer

Question

This is my setup. I'm trying to check every 5 epochs if I'm overfitting and using ADAM optimizer. And trying to save the trained network to later usages. Seems no matter the setup I can use ADAM optimizer it blow due to not being able to serialize. I tried the Momentum backpropagation and it works, but I would like to use ADAM.

System.out.println("LOAD TRAINING DATA.");
        DataSet<MLDataItem> trainingSet = new TabularDataSet(d.inputs[0].length, d.outputs[0].length);
        trainingSet.setColumnNames(d.headers);
        for (int i = 0; i < d.inputs.length; i++) {

            trainingSet.add(new TabularDataSet.Item(d.inputs[i], d.outputs[i]));
        }

        System.out.println("NORMALIZING TRAINING DATA.");
        DataSets.normalizeMax(trainingSet);
        //trainingSet.getColumnNames();

        System.out.println("CREATING NETWORK.");
        neuralNet = FeedForwardNetwork.builder()
                .addInputLayer(d.inputs[0].length)
                .addFullyConnectedLayer(d.inputs[0].length, ActivationType.SIGMOID)
                .addFullyConnectedLayer((int) d.inputs[0].length / 2, ActivationType.SIGMOID)
                .addFullyConnectedLayer((int) d.inputs[0].length / 4, ActivationType.SIGMOID)
                .addOutputLayer(d.outputs[0].length, ActivationType.SIGMOID)
                .lossFunction(LossType.MEAN_SQUARED_ERROR)
                .randomSeed(123)
                .build();

        System.out.println("TRAINING CONFIGURATIONS.");
        neuralNet.setLabel("TRAINING DATA");
        BackpropagationTrainer trainer = neuralNet.getTrainer();
        trainer.setBatchMode(false);

        trainer.setEarlyStopping(true);
        trainer.setEarlyStoppingMinLossChange(0.00000001F);
        trainer.setEarlyStoppingPatience(5);

        trainer.setLearningRate(0.001F);
        trainer.setMaxEpochs(100);
        trainer.setMaxError(0.0001F);
        trainer.setMomentum(0F);

        trainer.setTrainingSnapshots(true);

        trainer.setOptimizer(OptimizerType.ADAM);
        System.out.println("TRAINING...");
        neuralNet.train(trainingSet);
        neuralNetFile = "neuralNetwork_" + timeStamp + ".dnet";

        System.out.println("SAVING NETWORK INTO " + neuralNetFile);
        FileIO.writeToFile(neuralNet, neuralNetFile);
        System.out.println("DONE!!!");

NotSerializable Exception on the earlyStopping

    at DeepNettsLearning.train(DeepNettsLearning.java:100) [classes/:?]
    at DeepNettsLearning.main(DeepNettsLearning.java:36) [classes/:?]
Epoch:20, Time:2ms, TrainError:0.053135615, TrainErrorChange:-0.0012555942, TrainAccuracy: 0.9285714
Epoch:21, Time:4ms, TrainError:0.0519301, TrainErrorChange:-0.0012055151, TrainAccuracy: 0.9285714
Epoch:22, Time:4ms, TrainError:0.050770074, TrainErrorChange:-0.0011600256, TrainAccuracy: 0.9285714
Epoch:23, Time:3ms, TrainError:0.049651828, TrainErrorChange:-0.0011182465, TrainAccuracy: 0.9285714
Epoch:24, Time:2ms, TrainError:0.048572194, TrainErrorChange:-0.0010796338, TrainAccuracy: 0.9285714
Catching
java.io.NotSerializableException: deepnetts.net.train.opt.AdamOptimizer
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1193) ~[?:?]
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1579) ~[?:?]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1536) ~[?:?]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1444) ~[?:?]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1187) ~[?:?]
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1579) ~[?:?]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1536) ~[?:?]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1444) ~[?:?]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1187) ~[?:?]
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1579) ~[?:?]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1536) ~[?:?]
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1444) ~[?:?]
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1187) ~[?:?]
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1579) ~[?:?]
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1536) ~[?:?]

And also when saving to file

Total Training Time: 595ms
------------------------------------------------------------------------
SAVING NETWORK INTO neuralNetwork_2021.03.19.17.31.37.dnet
Exception in thread "main" java.io.NotSerializableException: deepnetts.net.train.opt.AdamOptimizer
    at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1193)
    at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1579)
    at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1536)
    at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1444)
    at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1187)
    at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1579)

Anyone can help me with this? Thanks in advance

Answer 1

Looks like DeepNetts have a bug in their code. I've checked their GitHub and there is no adam optimizer, so I assume you are using professional edition

Could you ask their support about this issue?