penalty algorithm

One of the ways of replacing a constrained optimization problem with an unconstrained one is by adding a penalty function to the objective function that depends - in some logical way - on the value of the constraints.

The idea is to minimize a sequence of unconstrained minimization problems where the infeasibility of the constraints is minimized together with the objective function.

Following text was taken from http://mdolab.utias.utoronto.ca/aer1415.html:

## 4.2 Penalty and Barrier Methods

One of the ways of replacing a constrained optimization problem with an unconstrained one is by adding a penalty function to the objective function that increases depending on the value of the constraints.

There two main types of penalization methods: penalty functions impose a penalty for violating a constrain, and barrier methods other imposes a penalty for reaching the boundary of an inequality constraint.

### 4.2.1 The Quadratic Penalty Method

This is the simplest type of penalty function. The new objective function will be the original one plus a term for each constraint, which is positive when the current point violates the constraint and zero otherwise.

Consider the equality-constrained problem:

minimize subject to where is an m-dimensional vector whose j-th component is . We assume that all functions are twice continuously differentiable.

We require a penalty for constraint violation to be a continuous function with the following properties

if is feasible

otherwise,

The new objective function is

, (4.22)

were is positive and is called the penalty parameter. The quadratic penalty function is defined as

#### Algorithm:

1. Check termination conditions. if satisfies the optimality conditions, he algorithm terminates successfully.
2. Minimize the penalty function. With as the starting point, execute an algorithm to solve the unconstrained subproblem minimize w.r.t. and let the solution of this subproblem be .
3. Increase the penalty parameter. Set to a larger value than , set k = k + 1 and return to 1.