How to measure time per layer in Chainer

Question

How can I measure the time taken by each layer? This should include both forward and backward pass.

For instance, in the case of VGG, I wanna know the time each of the layers takes. Part of the code is shown below.

   h = F.relu(self.conv1_2(h))
   h = F.max_pooling_2d(h, 2, 2)

   h = F.relu(self.conv2_1(h))
   h = F.relu(self.conv2_2(h))
   h = F.max_pooling_2d(h, 2, 2)

Answer 1

In case of the forward pass you may try to monkey-patch the execution of the model something like this:

import time 
from functools import wraps

def check_time(link_name, func):

    @wraps(func)
    def wrapper(*args, **kwargs):
        t = time.time()
        res = func(*arsg, **kwargs)
        t = time.time() - t
        print("Execution of {0} of the link {1} took {2:.3f} sec".format(func.__name__, link_name, t)
        return res

for name, link in model.namedlinks(skipself=True):
    link.forward = check_time(name, link.forward)

Since chainer follows the "define-by-run" strategy, the computation graph is constructed while you running the code, so the backward() method is only defined on chainer.Function instances. You would need to monkey-patch the according methods after each run, which would make you code quite slow, I guess.

Anyway, I hope, my code gives you an idea of how to do what you want.