Multi-layer Perceptron Networks - The Diigo Meta page

• 04 Jul 13
  

  chrispbailey

  neuralnetwork

• 17 Jan 13
  

  harry palmer

  anns tanh functions activation-functions

  • A nonlinear problem

      Consider again the best linear fit we found for the car data. Notice that the data points are not evenly distributed around the line: for low weights, we see more miles per gallon than our model predicts. In fact, it looks as if a simple curve might fit these data better than the straight line. We can enable our neural network to do such curve fitting by giving it an additional node which has a suitably curved (nonlinear) activation function. A useful function for this purpose is the S-shaped hyperbolic tangent (tanh) function (Fig. 1). 

      

     
• 19 Nov 11
  

  lbjrebecca
  • additional node which has a suitably curved (nonlinear) activation function
  • all non-input neural network units have such a bias weight.
  • bias weights shift the tanh function in the x- and y-direction
  • other two weights scale it along those two directions
  • too large a hidden layer - or too many hidden layers - can degrade the network's performance
  • any function can be expressed as a linear combination of tanh functions: tanh is a universal basis function
  • two classes of activation functions commonly used in neural networks are the sigmoidal (S-shaped) basis functions (to which tanh belongs), and the radial basis functions

  5 more annotations...

Would you like to comment?

Join Diigo for a free account, or sign in if you are already a member.