Artificial Neural network or ANN is a very popular method for predictive or optimization or simulation objectives. ANN mimics the human nervous system to solve problems in a parallel manner. ANN are known to be adaptable with situations, flexible with data and efficient enough for predicting any kind of problems. The limitation of ANN lies into the overdependence on data for learning the problem. Also there is no specific rule for the selection of activation function and number of hidden layers. However the application of ANN is still growing and various new forms of ANN is now utilized to solve problems from engineering, science as well as literature. The new methods mainly tries to solve the above discussed limitations by merging ANN with other or developing completely new algorithms.