A Gegenbauer Neural Network with Regularized Weights Direct Determination for Classification

by   Jie He, et al.

Single-hidden layer feed forward neural networks (SLFNs) are widely used in pattern classification problems, but a huge bottleneck encountered is the slow speed and poor performance of the traditional iterative gradient-based learning algorithms. Although the famous extreme learning machine (ELM) has successfully addressed the problems of slow convergence, it still has computational robustness problems brought by input weights and biases randomly assigned. Thus, in order to overcome the aforementioned problems, in this paper, a novel type neural network based on Gegenbauer orthogonal polynomials, termed as GNN, is constructed and investigated. This model could overcome the computational robustness problems of ELM, while still has comparable structural simplicity and approximation capability. Based on this, we propose a regularized weights direct determination (R-WDD) based on equality-constrained optimization to determine the optimal output weights. The R-WDD tends to minimize the empirical risks and structural risks of the network, thus to lower the risk of over fitting and improve the generalization ability. This leads us to a the final GNN with R-WDD, which is a unified learning mechanism for binary and multi-class classification problems. Finally, as is verified in the various comparison experiments, GNN with R-WDD tends to have comparable (or even better) generalization performances, computational scalability and efficiency, and classification robustness, compared to least square support vector machine (LS-SVM), ELM with Gaussian kernel.


Backward-Forward Algorithm: An Improvement towards Extreme Learning Machine

Extreme learning machine (ELM), a randomized learning paradigm for a sin...

Mixture of ELM based experts with trainable gating network

Mixture of experts method is a neural network based ensemble learning th...

A priori estimates for classification problems using neural networks

We consider binary and multi-class classification problems using hypothe...

Stochastic Configuration Networks: Fundamentals and Algorithms

This paper contributes to a development of randomized methods for neural...

Rank Based Pseudoinverse Computation in Extreme Learning Machine for Large Datasets

Extreme Learning Machine (ELM) is an efficient and effective least-squar...

Solution Path Algorithm for Twin Multi-class Support Vector Machine

The twin support vector machine and its extensions have made great achie...

Approximate Probabilistic Neural Networks with Gated Threshold Logic

Probabilistic Neural Network (PNN) is a feed-forward artificial neural n...

Please sign up or login with your details

Forgot password? Click here to reset