A characterization of the behavior of the Anderson acceleration on linear problems

We give a complete characterization of the behavior of the Anderson acceleration (with arbitrary nonzero mixing parameters) on linear problems. Let ν be the grade of the residual at the starting point with respect to the matrix defining the linear problem. We show that if Anderson acceleration does not stagnate (that is, produces different iterates) up to ν, then the sequence of its iterates converges to the exact solution of the linear problem. Otherwise, the Anderson acceleration converges to a point that is not a solution. Anderson acceleration and GMRES are essentially equivalent up to the index where the iterates of Anderson acceleration begin to stagnate. This result holds also for an optimized version of Anderson acceleration, where at each step the mixing parameter is chosen so that it minimizes the residual of the current iterate.