activation function design

This paper suggests that a simple modification to the initial search direction can also substantially improve the training efficiency of almost all major optimization methods. It was discovered that if the initial search direction is locally modified

Feedforward neural network structures have extensively been considered in the literature. In a significant volume of research and development studies hyperbolic tangent type of a neuronal nonlinearity has been utilized. This paper dwells on the w