How to declare per neuron connections in a pytorch model?

Question

I want to create a brain-alike mess:

• We have input tensor I of length n and output tensor O of length p
• In between, we have K "intersection" layers
• At intersection layers, neurons share their values at that time with a random subset of "close" neurons (+- range c). In the form of w_i*current_neuron_val
• and J "creation" layers new neurons are created from a set or ReLued "close" neurons (+- range c). closed neurons do not go into deeper layers.

Can we do such a thing with PyTorch so that such model will be trainable?

Answer 1

You should create your own class based on nn.Module and make realisation for forward and backward manually. I guess those brain-like-mess connections you have to control through feature labeling inside tensor (aka "0" - no connection). Because all frameworks use tensors - they by default fully connected and another behavior have to be inducation in the code.

see code of bindsNET - how they implement spike-like model using pytorch