@menu
* Functions and Variables for augmented_lagrangian::
@end menu

@node Functions and Variables for augmented_lagrangian,  , augmented_lagrangian, augmented_lagrangian
@section Functions and Variables for augmented_lagrangian

@deffn  {Function} augmented_lagrangian_method (@var{FOM}, @var{xx}, @var{C}, @var{yy})
@deffnx {Function} augmented_lagrangian_method (@var{FOM}, @var{xx}, @var{C}, @var{yy}, optional_args)
@deffnx {Function} augmented_lagrangian_method ([@var{FOM}, @var{grad}], @var{xx}, @var{C}, @var{yy})
@deffnx {Function} augmented_lagrangian_method ([@var{FOM}, @var{grad}], @var{xx}, @var{C}, @var{yy}, optional_args)

Returns an approximate minimum of the expression @var{FOM}
with respect to the variables @var{xx},
holding the constraints @var{C} equal to zero.
@var{yy} is a list of initial guesses for @var{xx}.
The method employed is the augmented Lagrangian method (see Refs [1] and [2]).

@var{grad}, if present, is the gradient of @var{FOM} with respect to @var{xx},
represented as a list of expressions,
one for each variable in @var{xx}.
If not present, the gradient is constructed automatically.

@var{FOM} and each element of @var{grad}, if present,
must be ordinary expressions, not names of functions or lambda expressions.

@code{optional_args} represents additional arguments,
specified as @code{@var{symbol} = @var{value}}.
The optional arguments recognized are:

@table @code
@item niter
Number of iterations of the augmented Lagrangian algorithm
@item lbfgs_tolerance
Tolerance supplied to LBFGS
@item iprint
IPRINT parameter (a list of two integers which controls verbosity) supplied to LBFGS
@item %lambda
Initial value of @code{%lambda} to be used for calculating the augmented Lagrangian
@end table

This implementation minimizes the augmented Lagrangian by
applying the limited-memory BFGS (LBFGS) algorithm,
which is a quasi-Newton algorithm.

@code{load(augmented_lagrangian)} loads this function.

See also @code{lbfgs}.

References:

@ifnottex
[1] @url{http://www-fp.mcs.anl.gov/otc/Guide/OptWeb/continuous/constrained/nonlinearcon/auglag.html}

[2] @url{http://www.cs.ubc.ca/spider/ascher/542/chap10.pdf}
@end ifnottex
@iftex
[1] @url{http://www-fp.mcs.anl.gov/@-otc/@-Guide/@-OptWeb/@-continuous/@-constrained/@*nonlinearcon/@-auglag.html}

[2] @url{http://www.cs.ubc.ca/spider/ascher/542/chap10.pdf}
@end iftex

Examples:

@c ===beg===
@c load (lbfgs);
@c load (augmented_lagrangian);
@c FOM: x^2 + 2*y^2;
@c xx: [x, y];
@c C: [x + y - 1];
@c yy: [1, 1];
@c augmented_lagrangian_method(FOM, xx, C, yy, iprint=[-1,0]);
@c ===end===
@example
(%i1) load (lbfgs);
(%o1)     /maxima/share/lbfgs/lbfgs.mac
(%i2) load (augmented_lagrangian);
(%o2) 
   /maxima/share/contrib/augmented_lagrangian.mac
(%i3) FOM: x^2 + 2*y^2;
                               2    2
(%o3)                       2 y  + x
(%i4) xx: [x, y];
(%o4)                        [x, y]
(%i5) C: [x + y - 1];
(%o5)                      [y + x - 1]
(%i6) yy: [1, 1];
(%o6)                        [1, 1]
(%i7) augmented_lagrangian_method(FOM, xx, C, yy, iprint=[-1,0]);
(%o7) [[x = 0.66665984108002, y = 0.33334027245545],
                                 %lambda = [- 1.333337940892525]]
@end example

Same example as before, but this time the gradient is supplied as an argument.

@c ===beg===
@c load (lbfgs)$
@c load (augmented_lagrangian)$
@c FOM: x^2 + 2*y^2;
@c xx: [x, y];
@c grad : [2*x, 4*y];
@c C: [x + y - 1];
@c yy: [1, 1];
@c augmented_lagrangian_method ([FOM, grad], xx, C, yy, 
@c                              iprint = [-1, 0]);
@c ===end===
@example
(%i1) load (lbfgs)$
(%i2) load (augmented_lagrangian)$
(%i3) FOM: x^2 + 2*y^2;
                               2    2
(%o3)                       2 y  + x
(%i4) FOM: x^2 + 2*y^2;
                               2    2
(%o4)                       2 y  + x
(%i5) xx: [x, y];
(%o5)                        [x, y]
(%i6) grad : [2*x, 4*y];
(%o6)                      [2 x, 4 y]
(%i7) C: [x + y - 1];
(%o7)                      [y + x - 1]
(%i8) yy: [1, 1];
(%o8)                        [1, 1]
(%i9) augmented_lagrangian_method ([FOM, grad], xx, C, yy,
                                   iprint = [-1, 0]);
(%o9) [[x = 0.666659841080025, y = .3333402724554462], 
                                 %lambda = [- 1.333337940892543]]
@end example

@opencatbox
@category{Numerical methods} @category{Optimization} @category{Share packages} @category{Package augmented_lagrangian} 
@closecatbox

@end deffn

