Tensor networks vs. Neural networks

Question

What are the differences between Tensor network theory vs. regular neural networks, where does the tensor nature come into place? After all a matrix is a rank 2 tensor. For me as layman both look the same: Input layer, Hidden layers, output layer:

It seems that Tensor networks is something much bigger, as it comes up also in theoretical physics.

Apparently I am missing something in the model.

Answer 1

Tensor network theory comes historically first as theory / mathematical model. Artificial Neural networks can be seen as an application of the above "borrowing" some of the concepts elaborated in such models while integrating and further elaborating.