TorchVision using pretrained weights for entire model vs backbone

Question

TorchVision Detection models have a weights and a weights_backbone parameter. Does using pretrained weights imply that the model uses pretrained weights_backbone under the hood? I am training a RetinaNet model and um unsure which of the two options I should use and what the differences are.

Answer 1

The difference is pretty simple: you can either choose to do transfer learning on the backbone only or on the whole network.

RetinaNet from Torchvision has a Resnet50 backbone. You should be able to do both of:

• retinanet_resnet50_fpn(weights=RetinaNet_ResNet50_FPN_Weights.COCO_V1)
• retinanet_resnet50_fpn(backbone_weights=ResNet50_Weights.IMAGENET1K_V1)

As implied by their names, the backbone weights are different. The former were trained on COCO (object detection) while the later were trained on ImageNet (classification).

To answer your question, pretrained weights implies that the whole network, including backbone weights, are initialized. However, I don't think that it calls backbone_weights under the hood.