A sequential algorithm for feed-forward neural networks with optimal coefficients and interacting frequencies

An algorithm for sequential approximation with optimal coefficients and interacting frequencies (SAOCIF) for feed-forward neural networks is presented. SAOCIF combines two key ideas. The first one is the optimization of the coefficients (the linear part of the approximation). The second one is the strategy to choose the frequencies (the non-linear weights), taking into account the interactions with the previously selected ones. The resulting method combines the locality of sequential approximations, where only one frequency is found at every step, with the globality of non-sequential methods, where every frequency interacts with the others. The idea behind SAOCIF can be theoretically extended to general Hilbert spaces. Experimental results show a very satisfactory performance.