TensorFlow : how to use Model.Predict for multiple predictions at once

62 views Asked by At

I'm using 5 different neural networks for voice/speech generation. I have 2 different networks for sound recognization/synthesis and 3 networks for tone/frequency recognization/synthesis. At the moment I'm using Model.Predict to make use of the networks after training and the training works 100%. But prediction for the networks (to make use of trained networks) is very slow: 10 ms / step / block for a single net. To compare: training costs 3 ms for 52 blocks. The problem is that when I run Model.Predict for 20 blocks instead of 1 block the output data contains errors. There's either almost no signal or it contains errors. I suspect that there's only one block processed instead of 20, but I'm not sure. This is a part of my code (for running 2 of the 5 nets), it's Pascal code because I use Keras4Delphi:

SetLength(xtestarray, BlocksCount*(SoundSize DIV 2)* 1);

for i := 0 to BlocksCount-1 do  
for j := 0 to (SoundSize DIV 2)-1 do
  xtestarray[i*BlocksCount+j] := InputFFT[i,j];

x_test := TNumPy.npArray<Double>(xtestarray);

x_test := TNDArray(x_test.reshape([BlocksCount, (SoundSize DIV 2),  1]));

y_test := model.Predict(x_test);

y_test := TNDArray(y_test.reshape([BlocksCount, SoundCount]));

x_test2 := y_test;

y_test2 := model2.Predict(x_test2);

So when I set BlocksCount on 1 (instead of 2 or 20 which I want), then the code works. I'm sure that when I'll use Model.Fit then the code will work for more than 1 BlockCount. Besides that Model.Fit is much faster. So is it possible to use Model.Fit without the connection weights being updated so without training? Or will Model.Predict work with some small changes? Then I'll use Model.Fit for 4 of the 5 nets because these nets must be updated anyway while the speech generation is used after training. But when it is possible for the only 1 net left (for sound recognization) to use Model.Fit without training that will also be fine.

0

There are 0 answers