What are the optimum hyperparameter settings (loss function, learning rate, data augmentation, learning rate scheduler, etc.) to achieve >75% accuracy with ImageNet50 (2012) dataset using ResNet50? I have tried the suggested parameters from here, but my accuracy is nowhere near 75% (it's below 30%) after 90 epochs. What I am missing?
currently, I am using the following settings-
import torch
import torchvision
batch_size = 256
epoch_n = 90 #number of epochs
#training data transformer for data augmentation
train_transform= torchvision.transforms.Compose([
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
#loss function, optimizer, and learning rate scheduler
criterion = torch.nn.CrossEntropyLoss(label_smoothing=0.1)
optimizer = torch.optim.SGD(net.parameters(), lr=batch_size/1000, momentum=0.875, weight_decay=1/32768)
lr_scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=epoch_n)