Linear vs nonlinear neural network?

For starters, a neural network can model any function (not just linear functions) Have a look at this – http://neuralnetworksanddeeplearning.com/chap4.html.

A Neural Network has got non linear activation layers which is what gives the Neural Network a non linear element.

The function for relating the input and the output is decided by the neural network and the amount of training it gets. If you supply two variables having a linear relationship, then your network will learn this as long as you don’t overfit. Similarly, a complex enough neural network can learn any function.

Leave a Comment