I am new to python and pytorch. I am struggling to understand the usage of Adam optimizer. Please review the below line of code:
opt = torch.optim.Adam([y], lr=0.1)
This line is taking a input list. Print the value of y, I get :
The value of [y]= [tensor([-0.7854, -0.7854, -0.7854], requires_grad=True)]
Now,when I print opt, I get:
The value of opt= Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
capturable: False
eps: 1e-08
foreach: None
lr: 0.1
maximize: False
weight_decay: 0
)
I am not able to understand the reason for finding opt and how it affects the code functionality?
Then further in the code, they do:
opt.zero_grad() # What is the use this function call?
loss, expval = cost(y) # This is a user defined function. I am understanding it
loss.backward() # This is a user defined function. I am understanding it
opt.step() # What is the use this function call?
What zero_grad() and step() function do and how are they dependent on opt? A small explanation can help me up a lot.
Thank you for great help.