An augmented Lagrangian approach with a variable transformation in nonlinear programming

Tangent cone and (regular) normal cone of a closed set under an invertible variable transformation around a given point are investigated, which lead to the concepts of θ−1θ−1-tangent cone of a set and θ−1θ−1-subderivative of a function. When the notion of θ−1θ−1-subderivative is applied to perturbation functions, a class of augmented Lagrangians involving an invertible mapping of perturbation variables are obtained, in which dualizing parameterization and augmenting functions are not necessarily convex in perturbation variables. A necessary and sufficient condition for the exact penalty representation under the proposed augmented Lagrangian scheme is obtained. For an augmenting function with an Euclidean norm, a sufficient condition (resp., a sufficient and necessary condition) for an arbitrary vector (resp., 0) to support an exact penalty representation is given in terms of θ−1θ−1-subderivatives. An example of the variable transformation applied to constrained optimization problems is given, which yields several exact penalization results in the literature.