Qscheme getting flipped in when quantizing convolution layers

105 views Asked by At

I'm experimenting with a few modifications to the quantization aware training. When I quantize my model by converting my qat conv2d layer to quantized conv2d module, the weight tensor is getting a qscheme of torch.per_channel_affine even though the qscheme of the fake quant object is torch.per_channel_symmetric. Below is a small code snippet surrounding this

weight_post_process(mod.weight)
wt_scale, wt_zp = weight_post_process.calculate_qparams()
wt_axis = weight_post_process.ch_axis
qweight = torch.quantize_per_channel(mod.weight.float(), wt_scale.to(torch.double), wt_zp.to(torch.int64), wt_axis, torch.qint8)
print(qweight)
print(weight_post_process.qscheme)

The printed qweight has an affine qscheme even though the fake quant has a symmetric qscheme

This causes an issue with

qconv = torch.nn.quantized.modules.conv.Conv2d(mod.in_channels, mod.out_channels, mod.kernel_size, mod.stride, mod.padding,
                                                      mod.dilation, mod.groups, mod.bias is not None, mod.padding_mode)
qconv.set_weight_bias(qweight, mod.bias)

The last line throws this error

...../site-packages/torch/_ops.py", line 143, in __call__
return self._op(*args, **kwargs or {})
RuntimeError: Unsupported qscheme: per_channel_affine

Any ideas on why the qscheme gets flipped and how I can prevent it?

0

There are 0 answers