I am trying to understand the tensorflow quantization package but have some issues with the understanding of the number of nodes added after creating the quantization model for quantization-aware-training procedure
Quantization tensorflow package, explanation of addition parameters in CNN kernel (27 for 12 3X3 kernels, and 5 for dense layer) Model summary Without Quantization model:
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
reshape (Reshape) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 26, 26, 12) 120
max_pooling2d (MaxPooling2 (None, 13, 13, 12) 0
D)
flatten (Flatten) (None, 2028) 0
dense (Dense) (None, 10) 20290
=================================================================
Total params: 20410 (79.73 KB)
Trainable params: 20410 (79.73 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
Model summary After adding quantization nodes:
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
quantize_layer (QuantizeLa (None, 28, 28) 3
yer)
quant_reshape (QuantizeWra (None, 28, 28, 1) 1
pperV2)
quant_conv2d (QuantizeWrap (None, 26, 26, 12) 147
perV2)
quant_max_pooling2d (Quant (None, 13, 13, 12) 1
izeWrapperV2)
quant_flatten (QuantizeWra (None, 2028) 1
pperV2)
quant_dense (QuantizeWrapp (None, 10) 20295
erV2)
=================================================================
Total params: 20448 (79.88 KB)
Trainable params: 20410 (79.73 KB)
Non-trainable params: 38 (152.00 Byte)
_________________________________________________________________
The number of differences in the parameters is not understandable.
I tried reading blog posts regarding this and what i understood is min/max and zeros of the equation needs to be found for the layers, which account for 3 constants but here the number of quantization variables changes as the type of the layer changes, so it is confusing.