penalty algorithm

One of the ways of replacing a constrained optimization problem with an unconstrained one is by adding a penalty function to the objective function that depends - in some logical way - on the value of the constraints.

The idea is to minimize a sequence of unconstrained minimization problems where the infeasibility of the constraints is minimized together with the objective function.

Following text was taken from http://mdolab.utias.utoronto.ca/aer1415.html:

One of the ways of replacing a constrained optimization problem with an unconstrained one is by adding a penalty function to the objective function that increases depending on the value of the constraints.

There two main types of penalization methods: penalty functions impose a penalty for violating a constrain, and barrier methods other imposes a penalty for reaching the boundary of an inequality constraint.

This is the simplest type of penalty function. The new objective function will be the original one plus a term for each constraint, which is positive when the current point violates the constraint and zero otherwise.

Consider the equality-constrained problem:

minimize subject to where is an m-dimensional vector whose j-th component is . We assume that all functions are twice continuously differentiable.

We require a penalty for constraint violation to be a continuous function with the following properties

if is feasible

otherwise,

The new objective function is

, (4.22)

were is positive and is called the penalty parameter. The quadratic penalty function is defined as

- Check termination conditions. if satisfies the optimality conditions, he algorithm terminates successfully.
- Minimize the penalty function. With as the starting point, execute an algorithm to solve the unconstrained subproblem minimize w.r.t. and let the solution of this subproblem be .
- Increase the penalty parameter. Set to a larger value than , set k = k + 1 and return to 1.

penalty_factor is a parameter that used for special gridding rules like leq, geq, mean, wmean, hist and other of this kind. These gridding rules adds some condition and causes surfit for using penalty algorithm. Each of these gridding rules have additional parameter called "penalty_factor". This parameter sets how fast the penalty parameter should be changed.

It is recomended to set penalty_factor from -3 to 3, but for most cases the default value for penalty_factor should be usable.

`surfit`: gridding and contouring software.