Regularized gap function as penalty term for constrained minimization problems

By using the regularized gap function for variational inequalities, Li and Peng introduced a new penalty function Pα(x) for the problem of minimizing a twice continuously differentiable function in closed convex subset of the n-dimensional space Rn. Under certain assumptions, they proved that the original constrained minimization problem is equivalent to unconstrained minimization of Pα(x). The main purpose of this paper is to give an in-depth study of those properties of the objective function that can be extended from the feasible set to the whole Rn by Pα(x). For example, it is proved that the objective function has bounded level sets (or is strongly coercive) on the feasible set if and only if Pα(x) has bounded level sets (or is strongly coercive) on Rn. However, the convexity of the objective function does not imply the convexity of Pα(x) when the objective function is not quadratic, no matter how small α is. Instead, the convexity of the objective function on the feasible set only implies the invexity of Pα(x) on Rn. Moreover, a characterization for the invexity of Pα(x) is also given.