An adaptive Hager-Zhang conjugate gradient method

Abstract

Based on a singular value study, lower and upper bounds for the condition number of the matrix which generates search directions of the Hager-Zhang conjugate gradient method are obtained. Then, based on the insight gained by our analysis, a modified version of the Hager-Zhang method is proposed, using an adaptive switch form the Hager-Zhang method to the Hestenes-Stiefel method when the mentioned condition number is large. Global convergence of the method is established for the uniformly convex objective functions when the line search fulfills the strong Wolfe conditions. Numerical comparisons between the implementations of the proposed method and the Hager-Zhang method is made on a set of unconstrained optimization test problems of the CUTEr collection, using the performance profile introduced by Dolan and  Mor{\'e}. Comparative testing results are reported.