Behavioral analysis of the leader particle during stagnation in a particle swarm optimization algorithm

Concept of the particle swarms emerged from a simulation of the collective behavior of social creatures. It gradually evolved into a powerful derivative-free optimization techniques, now known as Particle Swarm Optimization (PSO) for solving multi-dimensional, multi-modal, and non-convex optimization problems. The dynamics governing the movement of the particles in PSO has invoked a great deal of research interest over the last decade. Theoretical investigations of PSO has mostly focused on particle trajectories in the search space and the parameter-selection. This work looks into the PSO algorithm from the perspective of the leader particle and takes into account stagnation, a situation where particles are trapped at less coveted local optima, thus preventing them from reaching more coveted global optima. We show that the points sampled by the leader particle satisfy a simple mathematical relation which demonstrates that they lie on a specific line. We demonstrate the condition under which for certain values of the parameters, particles stick to exploring one side of the stagnation point only and ignore the other side, and also the case where both sides are explored. We also obtain information about the gradient of the objective function during stagnation in PSO. We provide a large number of machine simulations which support our claims over several ranges of the control parameters. This sheds light on possible modifications to the basic PSO algorithm which would help future researchers to work with even more efficient and state-of-the-art PSO variants.