Acceleration of sequential subspace optimization in Banach spaces by orthogonal search directions

A standard solution technique for linear operator equations of first kind is the Landweber scheme which is an iterative method that uses the negative gradient of the current residual as search direction, which is also called the Landweber direction. Though this method proves to be stable with respect to noisy data, it is known to be numerically slow for problems in Hilbert spaces and this behavior shows to be even worse in some Banach space settings. This is why the idea came up to use several search directions instead of the Landweber direction only which has led to the development of Sequential Subspace Optimization (SESOP) methods. This idea is related to the famous Conjugate Gradient (CG) techniques that are known to be amongst the most effective methods to solve linear equations in Hilbert spaces. Since CG methods decisively make use of the inner product structure, they have been inherently restricted to Hilbert spaces so far. SESOP methods in Banach spaces do not share the conjugacy property with CG methods. In this article we use the concept of generalized orthogonality in Banach spaces and apply metric projections to orthogonalize the current Landweber direction with respect to the search space of the last iteration. This leads to an accelerated SESOP method which is confirmed by various numerical experiments. Moreover, in Hilbert spaces our method coincides with the Conjugate Gradient Normal Error (CGNE) or Craig's method applied to the normal equation. We prove weak convergence to the exact solution. Furthermore we perform a couple of numerical tests on a linear problem involving a random matrix and on the problem of 2D computerized tomography where we use different ℓp-spaces. In all experiments the orthogonalization of the search space shows superior convergence properties compared to standard SESOP. This especially holds for p close to 1. Letting p→2 the more we recover the conjugacy property for the search directions and the more the convergence behaves independently of the size of the search space.