Sharing weights in parallel Convolutional Layer

Question

currently I am developing a new network using NiftyNet and would need some help.

I am trying to implement an Autofocus Layer [1] as proposed in the paper.

However, at a certain point, the Autofocus Layer needs to calculate K (K=4) parallel convolutions each using the same weights (w) and concatenates the four outputs afterwards.

Is there a way to create four parallel convolutional layer with each having the same weights in NiftyNet?

Thank you in advance.

[1] https://arxiv.org/pdf/1805.08403.pdf

Answer 1

The solution to this problem is as follows.

There is no restriction allowing you to use the same convolutional layer multiple times, each time with another input. This simulates the desired parallelism and solves the weight sharing issue, because there is only one convolutional layer.

However, using this approach doesn't solve the issue having different dilation rates in each parallel layer - we only have one convolutional layer for the weight sharing problem as mentioned above.

Note: it is the same operation either using a given tensor as input for a convolutional layer with dilation rate = 2 OR using a dilated tensor with rate = 2 as input for a convolutional layer with dilation rate = 1.

Therefore, creating K dilated tensors each with a different dilation rate and then using each of them as input for a single convolutional layer with dilation rate = 1 solves the problem having parallel layers each with a different dilation rate.

NiftyNet provides a class to create dilated tensors.