A Constructive Approach for Data-Driven Randomized Learning of Feedforward Neural Networks

by   Grzegorz Dudek, et al.

Feedforward neural networks with random hidden nodes suffer from a problem with the generation of random weights and biases as these are difficult to set optimally to obtain a good projection space. Typically, random parameters are drawn from an interval which is fixed before or adapted during the learning process. Due to the different functions of the weights and biases, selecting them both from the same interval is not a good idea. Recently more sophisticated methods of random parameters generation have been developed, such as the data-driven method proposed in Anon19, where the sigmoids are placed in randomly selected regions of the input space and then their slopes are adjusted to the local fluctuations of the target function. In this work, we propose an extended version of this method, which constructs iteratively the network architecture. This method successively generates new hidden nodes and accepts them if the training error decreases significantly. The threshold of acceptance is adapted to the current training stage. At the beginning of the training process only those nodes which lead to the largest error reduction are accepted. Then, the threshold is reduced by half to accept those nodes which model the target function details more accurately. This leads to faster convergence and more compact network architecture, as it includes only "significant" neurons. Several application examples are given which confirm this thesis.


page 1

page 2

page 3

page 4


Data-Driven Randomized Learning of Feedforward Neural Networks

Randomized methods of neural network learning suffer from a problem with...

Improving Randomized Learning of Feedforward Neural Networks by Appropriate Generation of Random Parameters

In this work, a method of random parameters generation for randomized le...

A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes

Neural networks with random hidden nodes have gained increasing interest...

Autoencoder based Randomized Learning of Feedforward Neural Networks for Regression

Feedforward neural networks are widely used as universal predictive mode...

An Insect-Inspired Randomly, Weighted Neural Network with Random Fourier Features For Neuro-Symbolic Relational Learning

Insects, such as fruit flies and honey bees, can solve simple associativ...

Sequential three-way decisions with a single hidden layer feedforward neural network

The three-way decisions strategy has been employed to construct network ...

AutoRWN: automatic construction and training of random weight networks using competitive swarm of agents

Random Weight Networks have been extensively used in many applications i...

Please sign up or login with your details

Forgot password? Click here to reset