Abstract

The Extreme Learning Machine is a single-hidden-layer feedforward learning algorithm, which has been successfully applied in regression and classification problems in different research fields. The traditional algorithm assigns random weights and biases in the hidden layer, and the Moore-Penrose inverse matrix in the regularized least-squares method is adopted to compute the weights of the output layer. Training speed, generalization ability, and robustness are the advantages that characterize this algorithm, but it has some shortcomings in solving highly nonlinear problems. The scientific community adopted non-iterative multilayer learning models as effective and efficient measures, starting with the Multilayer Extreme Learning Machine, which incorporates an unsupervised extreme learning Autoencoder into its architecture for feature mapping. Since the literature does not present an in-depth review of non-iterative multilayer models, this paper focuses on a current description of the evolution of multilayer models, which are grouped into random mappings, kernel-correntropy strategies, and conditional probability techniques. In addition to showing the mathematical fundamentals of each model, a list of databases widely used in training multilayer networks is included. Finally, we present a class of fast iterative algorithms called Shrinkage-Thresholding, which solve the minimization problem associated with an Autoencoder.