Smoothed L1/2 regularizer learning for split-complex valued neuro-fuzzy algorithm for TSK system and its convergence results

This paper investigates an evolving split-complex valued neuro-fuzzy (SCVNF) algorithm for Takagi-Sugeno-Kang (TSK) system. In a bid to avoid the contradiction between boundedness and analyticity, splitting technique is traditionally employed to independently process the real part and the imaginary part of weight parameters in the system, which doubles weight dimension and causes oversized structure. For improving efficiency of structural optimization, previous studies have revealed that L1/2-norm regularizer can be effective in such sparse tasks thus is regarded as a representative of Lq (0 < q < 1) regularizer. To eliminate oscillation phenomenon and stabilize training procedure, a smoothed L1/2 regularizer learning is facilitated by smoothing the original one at the origin flexibly. It is rigorously proved that the real-valued cost function is monotonic decreasing during learning course, and the sum of gradient norm trends closer to zero. Plus some very general condition, the weight sequence itself is also convergent to a fixed point. Experimental results for the SCVNF are demonstrated, which match the theoretical analysis.