Constraints. Links to Optimization Toolbox Documentation. Set Bounds. Gradients and Hessians. Vectorized Constraints.

2-6 2-6 2-6 2-7 2-7

Using GlobalSearch and MultiStart
How to Optimize with GlobalSearch and MultiStart. Problems You Can Solve with GlobalSearch and MultiStart. Outline of Steps. Determining Inputs for the Problem. Create a Problem Structure. Create a Solver Object. Set Start Points for MultiStart. Run the Solver. Examining Results. Single Solution. Multiple Solutions. Iterative Display. Global Output Structures. Example: Visualizing the Basins of Attraction. Output Functions for GlobalSearch and MultiStart. Plot Functions for GlobalSearch and MultiStart. How GlobalSearch and MultiStart Work. Multiple Runs of a Local Solver. Differences Between the Solver Objects. GlobalSearch Algorithm. MultiStart Algorithm. Bibliography. Improving Results. Can You Certify a Solution Is Global?. Refining the Start Points. Changing Options. 3-2 3-2 3-2 3-4 3-4 3-13 3-16 3-19 3-23 3-23 3-24 3-28 3-31 3-32 3-34 3-38 3-43 3-43 3-43 3-45 3-49 3-52 3-53 3-53 3-56 3-63
Reproducing Results. GlobalSearch and MultiStart Examples. Example: Finding Global or Multiple Local Minima. Example: Using Only Feasible Start Points. Example: Parallel MultiStart. Example: Isolated Global Minimum.
3-67 3-71 3-71 3-78 3-82 3-85

Using Direct Search

What Is Direct Search?. Performing a Pattern Search. Calling patternsearch at the Command Line. Using the Optimization Tool for Pattern Search. Example: Finding the Minimum of a Function Using the GPS Algorithm. Objective Function. Finding the Minimum of the Function. Plotting the Objective Function Values and Mesh Sizes. Pattern Search Terminology. Patterns. Meshes. Polling. Expanding and Contracting. How Pattern Search Polling Works. Context. Successful Polls. An Unsuccessful Poll. Displaying the Results at Each Iteration. More Iterations. Stopping Conditions for the Pattern Search. Searching and Polling 4-2 4-3 4-3 4-3
4-7 4-7 4-8 4-9 4-11 4-11 4-12 4-13 4-13 4-14 4-14 4-15 4-18 4-19 4-20 4-21 4-24
Definition of Search. How To Use a Search Method. Search Types. When to Use Search. Example: Search and Poll. Description of the Nonlinear Constraint Solver. Performing a Pattern Search Using the Optimization Tool GUI. Example: A Linearly Constrained Problem. Displaying Plots. Example: Working with a Custom Plot Function. Performing a Pattern Search from the Command Line. Calling patternsearch with the Default Options. Setting Options for patternsearch at the Command Line. Using Options and Problems from the Optimization Tool. Pattern Search Examples: Setting Options. Poll Method. Complete Poll. Example: Comparing the Efficiency of Poll Options. Using a Search Method. Mesh Expansion and Contraction. Mesh Accelerator. Using Cache. Setting Tolerances for the Solver. Constrained Minimization Using patternsearch. Vectorizing the Objective and Constraint Functions.

Product Overview on page 1-2 Example: Comparing Several Solvers on page 1-4 What Is Global Optimization? on page 1-12 Choosing a Solver on page 1-17

Product Overview

In this section. Types of Problems and Solvers on page 1-2 Key Features on page 1-2 Implementation Notes on page 1-3
Types of Problems and Solvers
Global Optimization Toolbox provides methods that search for global solutions to problems that contain multiple maxima or minima. It includes global search, multistart, pattern search, genetic algorithm, and simulated annealing solvers. You can use these solvers to solve optimization problems where the objective or constraint function is continuous, discontinuous, stochastic, does not possess derivatives, or includes simulations or black-box functions with undefined values for some parameter settings. Genetic algorithm and pattern search solvers support algorithmic customization. You can create a custom genetic algorithm variant by modifying initial population and fitness scaling options or by defining parent selection, crossover, and mutation functions. You can customize pattern search by defining polling, searching, and other functions.

Key Features

Interactive tools for defining and solving optimization problems and monitoring solution progress Global search and multistart solvers for finding single or multiple global optima Genetic algorithm solver that supports linear, nonlinear, and bound constraints Multiobjective genetic algorithm with Pareto-front identification, including linear and bound constraints Pattern search solver that supports linear, nonlinear, and bound constraints
Simulated annealing tools that implement a random search method, with options for defining annealing process, temperature schedule, and acceptance criteria Parallel computing support in multistart, genetic algorithm, and pattern search solver Custom data type support in genetic algorithm, multiobjective genetic algorithm, and simulated annealing solvers

Implementation Notes

Global Optimization Toolbox solvers repeatedly attempt to locate a global solution. However, no solver employs an algorithm that can certify a solution as global. You can extend the capabilities of Global Optimization Toolbox functions by writing your own files, by using them in combination with other toolboxes, or with the MATLAB or Simulink environments.
Example: Comparing Several Solvers
In this section. Function to Optimize on page 1-4 Four Solution Methods on page 1-5 Comparison of Syntax and Solutions on page 1-10

In this section. Local vs. Global Optima on page 1-12 Basins of Attraction on page 1-13

Local vs. Global Optima

Optimization is the process of finding the point that minimizes a function. More specifically: A local minimum of a function is a point where the function value is smaller than or equal to the value at nearby points, but possibly greater than at a distant point. A global minimum is a point where the function value is smaller than or equal to the value at all other feasible points.
Global minimum Local minimum
Generally, Optimization Toolbox solvers find a local optimum. (This local optimum can be a global optimum.) They find the optimum in the basin of attraction of the starting point. For more information, see Basins of Attraction on page 1-13. In contrast, Global Optimization Toolbox solvers are designed to search through more than one basin of attraction. They search in various ways: GlobalSearch and MultiStart generate a number of starting points. They then use a local solver to find the optima in the basins of attraction of the starting points.
ga uses a set of starting points (called the population) and iteratively generates better points from the population. As long as the initial population covers several basins, ga can examine several basins. simulannealbnd performs a random search. Generally, simulannealbnd accepts a point if it is better than the previous point. simulannealbnd occasionally accepts a worse point, in order to reach a different basin. patternsearch looks at a number of neighboring points before accepting one of them. If some neighboring points belong to different basins, patternsearch in essence looks in a number of basins at once.

Basins of Attraction

If an objective function f(x) is smooth, the vector f(x) points in the direction where f(x) decreases most quickly. The equation of steepest descent, namely

d x(t) = f ( x(t)), dt

yields a path x(t) that goes to a local minimum as t gets large. Generally, initial values x(0) that are close to each other give steepest descent paths that tend to the same minimum point. The basin of attraction for steepest descent is the set of initial values leading to the same local minimum. The following figure shows two one-dimensional minima. The figure shows different basins of attraction with different line styles, and it shows directions of steepest descent with arrows. For this and subsequent figures, black dots represent local minima. Every steepest descent path, starting at a point x(0), goes to the black dot in the basin containing x(0).

To use the custom plot function, select Custom function in the Plot functions pane and enter @psplotchange in the field to the right. To compare the custom plot with the best function value plot, also select Best function value.
Now, when you run the example, the pattern search tool displays the plots shown in the following figure.
Note that because the scale of the y-axis in the lower custom plot is logarithmic, the plot will only show changes that are greater than 0.
How the Plot Function Works
The plot function uses information contained in the following structures, which the Optimization Tool passes to the function as input arguments: optimvalues Structure containing the current state of the solver flag String indicating the current status of the algorithm The most important statements of the custom plot function, psplotchange.m, are summarized in the following table.
Custom Plot Function Statements Statement

persistent last_best

Description Creates the persistent variable last_best, the best objective function value in the previous generation. Persistent variables are preserved over multiple calls to the plot function. Sets up the plot before the algorithm starts. Sets best equal to the minimum objective function value. The field optimvalues.fval contains the objective function value in the current iteration. The variable best is the minimum objective function value. For a complete description of the fields of the structure optimvalues, see Structure of the Plot Functions on page 9-11. Sets the variable change to the best objective function value at the previous iteration minus the best objective function value in the current iteration. Plots the variable change at the current objective function value, for the current iteration contained inoptimvalues.iteration.
set(gca,'Yscale','log') best = min(optimvalues.fval)
change = last_best - best
plot(optimvalues.iteration, change, '.r')
Performing a Pattern Search from the Command Line
In this section. Calling patternsearch with the Default Options on page 4-42 Setting Options for patternsearch at the Command Line on page 4-44 Using Options and Problems from the Optimization Tool on page 4-46
Calling patternsearch with the Default Options
This section describes how to perform a pattern search with the default options.

where c(x) represents the nonlinear inequality constraints, ceq(x) represents the equality constraints, m is the number of nonlinear inequality constraints, and mt is the total number of nonlinear constraints. The Augmented Lagrangian Genetic Algorithm (ALGA) attempts to solve a nonlinear optimization problem with nonlinear constraints, linear constraints, and bounds. In this approach, bounds and linear constraints are handled separately from nonlinear constraints. A subproblem is formulated by combining the fitness function and nonlinear constraint function using the Lagrangian and the penalty parameters. A sequence of such optimization problems are approximately minimized using the genetic algorithm such that the linear constraints and bounds are satisfied. A subproblem formulation is defined as
where the components i of the vector are nonnegative and are known as Lagrange multiplier estimates. The elements si of the vector s are nonnegative
shifts, and is the positive penalty parameter. The algorithm begins by using an initial value for the penalty parameter (InitialPenalty). The genetic algorithm minimizes a sequence of the subproblem, which is an approximation of the original problem. When the subproblem is minimized to a required accuracy and satisfies feasibility conditions, the Lagrangian estimates are updated. Otherwise, the penalty parameter is increased by a penalty factor (PenaltyFactor). This results in a new subproblem formulation and minimization problem. These steps are repeated until the stopping criteria are met. For a complete description of the algorithm, see the following references: [1] Conn, A. R., N. I. M. Gould, and Ph. L. Toint. A Globally Convergent Augmented Lagrangian Algorithm for Optimization with General Constraints and Simple Bounds, SIAM Journal on Numerical Analysis, Volume 28, Number 2, pages 545572, 1991. [2] Conn, A. R., N. I. M. Gould, and Ph. L. Toint. A Globally Convergent Augmented Lagrangian Barrier Algorithm for Optimization with General Inequality Constraints and Simple Bounds, Mathematics of Computation, Volume 66, Number 217, pages 261288, 1997.
Genetic Algorithm Optimizations Using the Optimization Tool GUI
In this section. Introduction on page 5-30 Displaying Plots on page 5-30 Example: Creating a Custom Plot Function on page 5-32 Reproducing Your Results on page 5-35 Example: Resuming the Genetic Algorithm from the Final Population on page 5-35

Introduction

The Optimization Tool GUI is described in the chapter Optimization Tool in the Optimization Toolbox documentation. This section describes some places where there are some differences between the use of the genetic algorithm in the Optimization Tool and the use of other solvers.
The Plot functions pane, shown in the following figure, enables you to display various plots of the results of the genetic algorithm.
Select the check boxes next to the plots you want to display.

x = 0.0027 -0.0052

fval = 0.0068
To get more information about the performance of the genetic algorithm, you can call ga with the syntax
[x fval exitflag output population scores] = ga(@fitnessfcn, nvars)
Besides x and fval, this function returns the following additional output arguments: exitflag Integer value corresponding to the reason the algorithm terminated output Structure containing information about the performance of the algorithm at each generation population Final population scores Final scores See the ga reference page for more information about these arguments.
Setting Options for ga at the Command Line
You can specify any of the options that are available for ga by passing an options structure as an input argument to ga using the syntax
[x fval] = ga(@fitnessfun, nvars, [],[],[],[],[],[],[],options)
This syntax does not specify any linear equality, linear inequality, or nonlinear constraints. You create the options structure using the function gaoptimset.
options = gaoptimset(@ga)
This returns the structure options with the default values for its fields.
options = PopulationType: PopInitRange: PopulationSize: EliteCount: CrossoverFraction: ParetoFraction: MigrationDirection: MigrationInterval: MigrationFraction: Generations: TimeLimit: FitnessLimit: StallGenLimit: StallTimeLimit: TolFun: TolCon: InitialPopulation: InitialScores: InitialPenalty: PenaltyFactor: PlotInterval: CreationFcn: FitnessScalingFcn: SelectionFcn: CrossoverFcn: MutationFcn: DistanceMeasureFcn: HybridFcn: Display: PlotFcns: OutputFcns: Vectorized: UseParallel:
'doubleVector' [2x1 double] 0.8000 [] 'forward' 20 0.Inf -Inf 50 Inf 1.0000e-006 1.0000e-006 [] [] 1 @gacreationuniform @fitscalingrank @selectionstochunif @crossoverscattered {[1x1 function_handle] [] [] 'final' [] [] 'off' 'never'
The function ga uses these default values if you do not pass in options as an input argument. The value of each option is stored in a field of the options structure, such as options.PopulationSize. You can display any of these values by entering options. followed by the name of the field. For example, to display the size of the population for the genetic algorithm, enter
options.PopulationSize ans = 20
To create an options structure with a field value that is different from the default for example to set PopulationSize to 100 instead of its default value 20 enter

Raw Scores of Population Score 60 50

10 Sorted individuals

The following plot shows the scaled values of the raw scores using rank scaling.
Scaled Values Using Rank Scaling 4.3.2.1.0.5

Scaled value

Because the algorithm minimizes the fitness function, lower raw scores have higher scaled values. Also, because rank scaling assigns values that depend
only on an individuals rank, the scaled values shown would be the same for any population of size 20 and number of parents equal to 32.
Comparing Rank and Top Scaling
To see the effect of scaling, you can compare the results of the genetic algorithm using rank scaling with one of the other scaling options, such as Top. By default, top scaling assigns 40 percent of the fittest individuals to the same scaled value and assigns the rest of the individuals to value 0. Using the default selection function, only 40 percent of the fittest individuals can be selected as parents. The following figure compares the scaled values of a population of size 20 with number of parents equal to 32 using rank and top scaling.
Comparison of Rank and Top Scaling 6 Scaled value 10 Sorted individuals Rank scaling Top scaling
Because top scaling restricts parents to the fittest individuals, it creates less diverse populations than rank scaling. The following plot compares the
variances of distances between individuals at each generation using rank and top scaling.
Variance of Distance Between Individuals Using Rank and Top Scaling 0.8 0.7 0.6 0.5 Variance 0.4 0.3 0.2 0.Variance using rank scaling Variance using top scaling

Selection

The selection function chooses parents for the next generation based on their scaled values from the fitness scaling function. An individual can be selected more than once as a parent, in which case it contributes its genes to more than one child. The default selection option, Stochastic uniform, lays out a line in which each parent corresponds to a section of the line of length proportional to its scaled value. The algorithm moves along the line in steps of equal size. At each step, the algorithm allocates a parent from the section it lands on. A more deterministic selection option is Remainder, which performs two steps: In the first step, the function selects parents deterministically according to the integer part of the scaled value for each individual. For example, if an individuals scaled value is 2.3, the function selects that individual twice as a parent.
In the second step, the selection function selects additional parents using the fractional parts of the scaled values, as in stochastic uniform selection. The function lays out a line in sections, whose lengths are proportional to the fractional part of the scaled value of the individuals, and moves along the line in equal steps to select the parents. Note that if the fractional parts of the scaled values all equal 0, as can occur using Top scaling, the selection is entirely deterministic.

ga, gamultiobj

Population evaluated in parallel, which occurs once per iteration. For more details, see Genetic Algorithm on page 8-8 and How to Use Parallel Processing on page 8-11. No vectorization of fitness or constraint functions.

Parallel?

Parallel Characteristics Poll points evaluated in parallel, which occurs once per iteration. For more details, see Pattern Search on page 8-7 and How to Use Parallel Processing on page 8-11. No vectorization of fitness or constraint functions.
No parallel functionality. However, simulannealbnd can use a hybrid function that runs in parallel. See Simulated Annealing on page 8-10.
In addition, several solvers have hybrid functions that run after they finish. Some hybrid functions can run in parallel. Also, most patternsearch search methods can run in parallel. For more information, see Parallel Search Functions or Hybrid Functions on page 8-14.
How Toolbox Functions Distribute Processes
parfor Characteristics and Caveats on page 8-3 MultiStart on page 8-5 GlobalSearch on page 8-6 Pattern Search on page 8-7 Genetic Algorithm on page 8-8 Parallel Computing with gamultiobj on page 8-9 Simulated Annealing on page 8-10
parfor Characteristics and Caveats
No Nested parfor Loops. Solvers employ the Parallel Computing Toolbox parfor function to perform parallel computations. Note parfor does not work in parallel when called from within another parfor loop.
Suppose, for example, your objective function userfcn calls parfor, and you want to call fmincon using MultiStart and parallel processing. Suppose also that the conditions for parallel gradient evaluation of fmincon are satisfied, as given in Parallel Optimization Functionality. The figure When parfor Runs In Parallel on page 8-4 shows three cases:
1 The outermost loop is parallel MultiStart. Only that loop runs in parallel. 2 The outermost parfor loop is in fmincon. Only fmincon runs in parallel. 3 The outermost parfor loop is in userfcn. In this case, userfcn can use

parfor in parallel.

Bold indicates the function that runs in parallel
1. problem = createOptimProblem(fmincon,'objective',@userfcn,.) ms = MultiStart('UseParallel','always'); x = run(ms,problem,10) Only the outermost parfor loop. runs in parallel If fmincon UseParallel option = 'always' fmincon estimates gradients in parallel
. x = fmincon(@userfcn,.). If fmincon UseParallel option = 'never' userfcn can use parfor in parallel. x = fmincon(@userfcn,.).
When parfor Runs In Parallel

MultiStart Properties

The UseParallel property determines whether the solver distributes start points to multiple processors: 'never' (default) Do not run in parallel. 'always' Run in parallel. For the solver to run in parallel you must set up a parallel environment with matlabpool. For details, see How to Use Parallel Processing on page 8-11.

Pattern Search Options

In this section. Optimization Tool vs. Command Line on page 9-9 Plot Options on page 9-10 Poll Options on page 9-12 Search Options on page 9-14 Mesh Options on page 9-19 Constraint Parameters on page 9-20 Cache Options on page 9-21 Stopping Criteria on page 9-21 Output Function Options on page 9-22 Display to Command Window Options on page 9-24 Vectorize and Parallel Options (User Function Evaluation) on page 9-25 Options Table for Pattern Search Algorithms on page 9-26
Optimization Tool vs. Command Line
There are two ways to specify options for pattern search, depending on whether you are using the Optimization Tool or calling the function patternsearch at the command line: If you are using the Optimization Tool, you specify the options by selecting an option from a drop-down list or by entering the value of the option in the text field. If you are calling patternsearch from the command line, you specify the options by creating an options structure using the function psoptimset, as follows:
options = psoptimset('Param1',value1,'Param2',value2,.);
See Setting Options for patternsearch at the Command Line on page 4-44 for examples.
In this section, each option is listed in two ways: By its label, as it appears in the Optimization Tool By its field name in the options structure For example: Poll method refers to the label of the option in the Optimization Tool. PollMethod refers to the corresponding field of the options structure.

Plot Options

Plot options enable you to plot data from the pattern search while it is running. When you select plot functions and run the pattern search, a plot window displays the plots on separate axes. You can stop the algorithm at any time by clicking the Stop button on the plot window. Plot interval (PlotInterval) specifies the number of iterations between consecutive calls to the plot function. You can select any of the following plots in the Plot functions pane. Best function value (@psplotbestf) plots the best objective function value. Function count (@psplotfuncount) plots the number of function evaluations. Mesh size (@psplotmeshsize) plots the mesh size. Best point (@psplotbestx) plots the current best point. Max constraint (@psplotmaxconstr) plots the maximum nonlinear constraint violation. Custom enables you to use your own plot function. To specify the plot function using the Optimization Tool,

options

Options structure. Create this structure with optimset, or by exporting from the Optimization Tool.

Vector of upper bounds.

A vector, a potential starting point for the optimization. Gives the dimensionality of the problem.
Vector of data points for lsqcurvefit.
Output Arguments Examples

problem

Optimization problem structure. Create a problem structure using Rosenbrocks function as objective (see Using a Hybrid Function on page 5-77), the interior-point algorithm for fmincon, and bounds with absolute value 2:
anonrosen = @(x)(100*(x(2) - x(1)^2)^2 + (1-x(1))^2); opts = optimset('Algorithm','interior-point'); problem = createOptimProblem('fmincon','x0',randn(2,1),. 'objective',anonrosen,'lb',[-2;-2],'ub',[2;2],. 'options',opts);

Alternatives

You can create a problem structure by exporting from the Optimization Tool (optimtool), as described in Exporting from the Optimization Tool on page 3-8.
optimtool | MultiStart | GlobalSearch

See Also Tutorials

Create a Problem Structure on page 3-4
Purpose Description Construction
User-supplied start points An object wrapper of a matrix whose rows represent start points for MultiStart.
tpoints = CustomStartPointSet(ptmatrix) generates a CustomStartPointSet object from the ptmatrix matrix. Each row of ptmatrix represents one start point. DimStartPoints

Properties

Dimension of each start point, a read-only property. DimStartPoints is the number of columns in ptmatrix.
DimStartPoints should be the same as the number of elements in problem.x0, the problem structure you pass to run. NumStartPoints
Number of start points, a read-only property. This is the number of rows in ptmatrix.

Methods

List custom start points in set

Copy Semantics Examples

Value. To learn how value classes affect copy operations, see Copying Objects in the MATLAB Programming Fundamentals documentation. Create a CustomStartPointSet object with 40 three-dimensional rows. Each row represents a normally distributed random variable with mean [10,10,10] and variance diag([4,4,4]):
fortypts = 10*ones(40,3) + 4*randn(40,3); % a matrix startpts = CustomStartPointSet(fortypts); startpts is the fortypts matrix in an object wrapper.
Get the fortypts matrix from the startpts object of the previous example:

x = ga(fitnessfcn,nvars,A,b,Aeq,beq,LB,UB) defines a set of lower and upper bounds on the design variables, x, so that a solution is found
in the range LB x UB. Use empty matrices for LB and UB if no bounds exist. Set LB(i) = -Inf if x(i) is unbounded below; set UB(i) = Inf if x(i) is unbounded above.
x = ga(fitnessfcn,nvars,A,b,Aeq,beq,LB,UB,nonlcon) subjects the minimization to the constraints defined in nonlcon. The function nonlcon accepts x and returns the vectors C and Ceq, representing the nonlinear inequalities and equalities respectively. ga minimizes the fitnessfcn such that C(x) 0 and Ceq(x)=0. (Set LB=[] and UB=[] if no bounds exist.)
Note that the nonlinear constraints are not satisfied when the PopulationType option is set to 'bitString' or 'custom'.
x = ga(fitnessfcn,nvars,A,b,Aeq,beq,LB,UB,nonlcon,options)
minimizes with the default optimization parameters replaced by values in the structure options, which can be created using the gaoptimset function. See the gaoptimset reference page for details.
x = ga(problem) finds the minimum for problem, where problem is a structure containing the following fields: fitnessfcn nvars Aineq Bineq Aeq Beq lb ub nonlcon rngstate solver options
Fitness function Number of design variables
A matrix for linear inequality constraints b vector for linear inequality constraints A matrix for linear equality constraints b vector for linear equality constraints
Lower bound on x Upper bound on x Nonlinear constraint function Optional field to reset the state of the random number generator
Options structure created using gaoptimset or the Optimization Tool
Create the structure problem by exporting a problem from Optimization Tool, as described in Importing and Exporting Your Work in the Optimization Toolbox documentation.
[x,fval] = ga(.) returns fval, the value of the fitness function
[x,fval,exitflag] = ga(.) returns exitflag, an integer
identifying the reason the algorithm terminated. The following lists the values of exitflag and the corresponding reasons the algorithm terminated. 1 Average cumulative change in value of the fitness function over options.StallGenLimit generations less than options.TolFun and constraint violation less than options.TolCon.
2 Fitness limit reached and constraint violation less than options.TolCon. 3 The value of the fitness function did not change in options.StallGenLimit generations and constraint violation less than options.TolCon. 4 Magnitude of step smaller than machine precision and constraint violation less than options.TolCon. 0 Maximum number of generations exceeded. -1 Optimization terminated by the output or plot function. -2 No feasible point found. -4 Stall time limit exceeded. -5 Time limit exceeded.

CreationFcn

CrossoverFcn

CrossoverFraction

The fraction of the population at the next generation, not including elite children, that is created by the crossover function Level of display Handle to the function that computes distance measure of individuals, computed in decision variable or design space (genotype) or in function space (phenotype)

Positive scalar | {0.8}

Display DistanceMeasureFcn
'off' | 'iter' | 'diagnose' | {'final'} {@distancecrowding,'phenotype'}

EliteCount

Description Positive integer specifying how many individuals in the current generation are guaranteed to survive to the next generation. Not used in gamultiobj. Scalar. If the fitness function attains the value of FitnessLimit, the algorithm halts. Handle to the function that scales the values of the fitness function Positive integer specifying the maximum number of iterations before the algorithm halts Handle to a function that continues the optimization after ga terminates or Cell array specifying the hybrid function and its options structure
Values Positive integer | {2}

FitnessLimit

Scalar | {-Inf}

FitnessScalingFcn

@fitscalingshiftlinear | @fitscalingprop | @fitscalingtop | {@fitscalingrank}

Generations

Positive integer |{100}

HybridFcn

Function handle | @fminsearch | @patternsearch | @fminunc | @fmincon | {[]} or 1-by-2 cell array | {@solver, hybridoptions}, where solver = fminsearch, patternsearch, fminunc, or fmincon {[]} Positive scalar | {10} Matrix | {[]}
InitialPenalty InitialPopulation
Initial value of penalty parameter Initial population used to seed the genetic algorithm; can be partial

InitialScores

Description Initial scores used to determine fitness; can be partial Direction of migration Scalar between 0 and 1 specifying the fraction of individuals in each subpopulation that migrates to a different subpopulation Positive integer specifying the number of generations that take place between migrations of individuals between subpopulations Handle to the function that produces mutation children Functions that ga calls at each iteration Scalar between 0 and 1 specifying the fraction of individuals to keep on the first Pareto front while the solver selects individuals from higher fronts Penalty update parameter
Values Column vector | {[]}
MigrationDirection MigrationFraction

'both' | {'forward'}

Scalar | {0.2}

MigrationInterval

Positive integer | {20}

MutationFcn

@mutationuniform | @mutationadaptfeasible | {@mutationgaussian}
OutputFcns ParetoFraction

Using a Search Method on page 4-61 Mesh Expansion and Contraction on page 4-65 Mesh Accelerator on page 4-71 Using Cache on page 4-74 Constrained Minimization Using patternsearch on page 4-79 Example of Vectorized Objective and Constraints on page 4-86
Example: Rastrigins Function on page 5-8 Example: Creating a Custom Plot Function on page 5-32 Example: Resuming the Genetic Algorithm from the Final Population on page 5-35 Example: Setting the Initial Range on page 5-51 Example: Linearly Constrained Population and Custom Plot Function on page 5-54 Example: Global vs. Local Minima with GA on page 5-73 Using a Hybrid Function on page 5-77 Constrained Minimization Using ga on page 5-83 Example: Multiobjective Optimization on page 7-7
Example: Minimizing De Jongs Fifth Function on page 6-8
accelerator mesh 4-71 algorithm genetic 5-21 pattern search 4-14 simulated annealing 6-13 annealing 6-12 annealing parameter 6-12
vectorizing 5-82 fitness scaling 5-60 Function files writing 2-2
ga function 12-7 gamultiobj function 12-13 gaoptimget function 12-20 gaoptimset function 12-21
cache 4-74 children crossover 5-23 elite 5-23 in genetic algorithms 5-20 mutation 5-23 constraint function vectorizing 4-82 creation function 9-37 linear feasible 5-54 range example 5-51 crossover 5-64 children 5-23 fraction 5-67
generations 5-19 genetic algorithm description 5-21 nonlinear constraint algorithm, ALGA 5-28 options 9-31 overview 5-2 setting options at command line 5-41 stopping criteria 5-25 using from command line 5-40 global and local minima 5-73 Global output function 3-35 GPS 4-2

hybrid function 5-77

direct search 4-2 diversity 5-19
individuals applying the fitness function 5-18 initial population 5-22
elite children 5-23 expansion mesh 4-65
MADS 4-2 maximizing functions 2-5 mesh 4-12 accelerator 4-71 expansion and contraction 4-65

fitness function 5-18

Index-1
minima global 5-73 local 5-73 minimizing functions 2-5 multiobjective optimization 7-2 mutation 5-64 options 5-65
noninferior solution 7-3 nonlinear constraint pattern search 4-43 nonlinear constraint algorithms ALGA 5-28 ALPS 4-30

Defining the Problem in Toolbox Syntax
To use Optimization Toolbox software, you need to
1 Define your objective function in the MATLAB language, as a function file
or anonymous function. This example will use a function file.
2 Define your constraint(s) as a separate file or anonymous function.
Function File for Objective Function
A function file is a text file containing MATLAB commands with the extension.m. Create a new function file in any text editor, or use the built-in MATLAB Editor as follows:
1 At the command line enter:

edit rosenbrock

The MATLAB Editor opens.

2 In the editor enter:

function f = rosenbrock(x) f = 100*(x(2) - x(1)^2)^2 + (1 - x(1))^2;
3 Save the file by selecting File > Save.
File for Constraint Function
Constraint functions must be formulated so that they are in the form
c(x)0 or ceq(x)=0. The constraint x1 + xneeds to be reformulated as x1 + x0 in order to have the correct syntax.
Furthermore, toolbox functions that accept nonlinear constraints need to have both equality and inequality constraints defined. In this example there is only an inequality constraint, so you must pass an empty array [] as the equality constraint function ceq. With these considerations in mind, write a function file for the nonlinear constraint:
1 Create a file named unitdisk.m containing the following code:
function [c, ceq] = unitdisk(x) c = x(1)^2 + x(2)^2 - 1; ceq = [ ];
2 Save the file unitdisk.m.

Running the Optimization

There are two ways to run the optimization: Using the Optimization Tool on page 1-7 Graphical User Interface (GUI) Using command line functions; see Minimizing at the Command Line on page 1-11.
1 Start the Optimization Tool by typing optimtool at the command line.

The following GUI opens.

For more information about this tool, see Chapter 5, Optimization Tool.
2 The default Solver fmincon - Constrained nonlinear minimization
is selected. This solver is appropriate for this problem, since Rosenbrocks function is nonlinear, and the problem has a constraint. For more information about how to choose a solver, see Choosing a Solver on page 2-4.

This table might differ from yours depending on toolbox version and computing platform. The following description applies to the table as displayed. The first column, labeled Iter, is the iteration number from 0 to 19. fmincon took 19 iterations to converge. The second column, labeled F-count, reports the cumulative number of times Rosenbrocks function was evaluated. The final row shows an F-count of 79, indicating that fmincon evaluated Rosenbrocks function 79 times in the process of finding a minimum. The third column, labeled f(x), displays the value of the objective function. The final value, 0.0456766, is the minimum that is reported in the Optimization Tool Run solver and view results box, and at the end of the exit message in the command window. The fourth column, Max constraint, goes from a value of 1 at the initial value, to very nearly 0, 1.474e005, at the final iteration. This column shows the value of the constraint function unitdisk at each iteration. Since
the value of unitdisk was nearly 0 at the final iteration, x1 + xthere.
The other columns of the iteration table are described in Iterative Display on page 3-15.
Example: Linear Programming
In this section. Example of Converting to Solver Form on page 1-14 Model Description on page 1-15 Solution Method on page 1-16 Bibliography on page 1-24
Example of Converting to Solver Form
This example demonstrates how to convert a problem from mathematical form into Optimization Toolbox solver syntax. While the problem is a linear program, the techniques apply to all solvers. The variables and expressions in the problem represent a model of operating a chemical plant, from an example in Edgar and Himmelblau [1]. There are two video demos that describe the problem. Optimization Modeling 1 shows the problem in pictorial form. It demonstrates how to generate the mathematical expressions of Model Description on page 1-15 from the picture. To view the demo, enter the following command at the MATLAB command line:
playbackdemo('Optimization-Modeling-1','toolbox/optim/web/demos');
Optimization Modeling 2 describes how to convert these mathematical expressions into Optimization Toolbox solver syntax. This video shows how to solve the problem, and how to interpret the results. To view the demo, enter the following command at the MATLAB command line:

min f ( x),

possibly subject to constraints. f(x) is called an objective function. In general, f(x) is a scalar function of type double, and x is a vector or scalar of type double. However, multiobjective optimization, equation solving, and some sum-of-squares minimizers, can have vector or matrix objective functions F(x)
of type double. To use Optimization Toolbox solvers for maximization instead of minimization, see Maximizing an Objective on page 2-27. Write the objective function for a solver in the form of a function file or anonymous function handle. You can supply a gradient f(x) for many solvers, and you can supply a Hessian for several solvers. See Writing Objective Functions on page 2-16. Constraints have a special form, as described in Writing Constraints on page 2-29.

Choosing a Solver

In this section. Optimization Decision Table on page 2-4 Choosing the Algorithm on page 2-7 Problems Handled by Optimization Toolbox Functions on page 2-12
Optimization Decision Table
The following table is designed to help you choose a solver. It does not address multiobjective optimization or equation solving. There are more details on all the solvers in Problems Handled by Optimization Toolbox Functions on page 2-12. Use the table as follows:
1 Identify your objective function as one of five types:
Linear Quadratic Sum-of-squares (Least squares) Smooth nonlinear Nonsmooth
2 Identify your constraints as one of five types:
None (unconstrained) Bound Linear (including bound) General smooth Discrete (integer)
3 Use the table to identify a relevant solver.

In this table:

Blank entries means there is no Optimization Toolbox solver specifically designed for this type of problem. * means relevant solvers are found in Global Optimization Toolbox functions (licensed separately from Optimization Toolbox solvers). fmincon applies to most smooth objective functions with smooth constraints. It is not listed as a preferred solver for least squares or linear or quadratic programming because the listed solvers are usually more efficient. The table has suggested functions, but it is not meant to unduly restrict your choices. For example, fmincon is known to be effective on some nonsmooth problems. The Global Optimization Toolbox ga function can be programmed to address discrete problems. It is not listed in the table because additional programming is needed to solve discrete problems.

options = optimset('FunValCheck','on');
3. Use Appropriate Tolerances
Solvers can fail to converge if tolerances are too small, especially TolFun and TolX. To change tolerances using the Optimization Tool, use the Stopping criteria list at the top of the Options pane. To change tolerances at the command line, use optimset as described in Setting Options on page 2-43.

4. Use a Plot Function

You can obtain more visual or detailed information about solver iterations using a plot function. For a list of the predefined plot functions, see Options > Plot functions in the Optimization Tool. The Options section of your solvers function reference pages also lists the plot functions. To use a plot function: Using the Optimization Tool, check the boxes next to each plot function you wish to use. At the MATLAB command line, enter
options = optimset('PlotFcns',{@plotfcn1,@plotfcn2,.});
Call the solver using the options structure. For an example of using a plot function, see Example: Using a Plot Function on page 3-29.
5. Enable DerivativeCheck
If you have supplied derivatives (gradients or Jacobians) to your solver, the solver can fail to converge if the derivatives are inaccurate. For more information about using the DerivativeCheck option, see Checking Validity of Gradients or Jacobians on page 2-50.
6. Use an Output Function
You can obtain detailed information about solver iterations using an output function. Solvers call output functions at each iteration. You write output functions using the syntax described in Output Function on page 9-18. For an example of using an output function, see Example: Using Output Functions on page 3-35.
7. Use a Sparse Solver or a Multiply Function
Large problems can cause MATLAB to run out of memory or time. Here are some suggestions for using less memory:
Use a large-scale algorithm if possible. These algorithms include trust-region-reflective algorithms, the fmincon interior-point algorithm, any LargeScale algorithm, the fsolve trust-region-dogleg algorithm, and the Levenberg-Marquardt algorithm. Use sparse matrices for your linear constraints. Use a Jacobian multiply function or Hessian multiply function. For examples, see Example: Jacobian Multiply Function with Linear Least Squares on page 6-162, Example: Quadratic Minimization with a Dense but Structured Hessian on page 6-126, and Example: Nonlinear Minimization with a Dense but Structured Hessian and Equality Constraints on page 6-68.
8. Use Parallel Computing
If you have a Parallel Computing Toolbox license, your solver might run faster using parallel computing. For more information, see Chapter 7, Parallel Computing for Optimization.
When the Solver Might Have Succeeded
In this section. Final Point Equals Initial Point on page 4-14 Local Minimum Possible on page 4-14

options = optimset('LargeScale','off'); funh = @(x)log(1 + (x(1) - 4/3)^2 + 3*(x(2) - (x(1)^3 - x(1)))^2); [xfinal fval exitflag] = fminunc(funh,[-1;2],options) Local minimum possible. fminunc stopped because it cannot decrease the objective function along the current search direction. xfinal = 1.3333 1.0370 fval =

8.5265e-014 exitflag = 5

The exit flag value of 5 indicates that the first-order optimality measure was above the function tolerance. Run the minimization again starting from xfinal:
[xfinal2 fval2 exitflag2] = fminunc(funh,xfinal,options) Local minimum found. Optimization completed because the size of the gradient is less than the default value of the function tolerance. xfinal2 = 1.3333 1.0370 fval2 = 6.5281e-014 exitflag2 = 1
The local minimum is found, not possible, and the exitflag is 1, not 5. The two solutions are virtually identical. Yet the second run has a more satisfactory exit message, since the first-order optimality measure was low enough: 7.5996e-007, instead of 3.9674e-006.
3. Try a Different Algorithm
Many solvers give you a choice of algorithm. Different algorithms can lead to the use of different stopping criteria. For example, Rerun Starting At Final Point returns exitflag 5 from the first run. This run uses the medium-scale algorithm. The large-scale algorithm requires a user-supplied gradient. betopt.m contains a calculation of the objective function and gradient:
function [f gradf] = betopt(x) g = 1 + (x(1)-4/3)^2 + 3*(x(2) - (x(1)^3-x(1)))^2; f = log(g); gradf(1) = 2*(x(1)-4/3) + 6*(x(2) - (x(1)^3-x(1)))*(1-3*x(1)^2); gradf(1) = gradf(1)/g; gradf(2) = 6*(x(2) - (x(1)^3 -x(1)))/g;
Running the optimization using the large-scale algorithm results in a different exitflag:
options = optimset('GradObj','on'); [xfinal3 fval3 exitflag3] = fminunc(@betopt,[-1;2],options) Local minimum possible. fminunc stopped because the final change in function value relative to its initial value is less than the default value of the function tolerance. xfinal3 = 1.3333 1.0370 fval3 = 7.6659e-012 exitflag3 = 3
The exit condition is better than the medium-scale condition, though it is still not the best. Rerunning the algorithm from the final point produces a better point, with extremely small first-order optimality measure:
[xfinal4 fval4 eflag4 output4] = fminunc(@betopt,xfinal3,options) Local minimum found. Optimization completed because the size of the gradient is less than the default value of the function tolerance.

optimtool(options)

Select File > Import Options in the Optimization Tool. Select File > Import Problem in the Optimization Tool. The methods described above require that the options, or problem and options, be present in the MATLAB workspace. If you import a problem that was generated with the Include information needed to resume this run box checked, the initial point is the latest point generated in the previous run. (For Genetic Algorithm solvers, the initial population is the latest population generated in the previous run. For the Simulated Annealing solver, the initial point is the best point generated in the previous run.) If you import a problem that was generated with this box unchecked, the initial point (or population) is the initial point (or population) of the previous run.

Generating a File

You may want to generate a file to continue with your optimization problem in the Command Window at another time. You can run the file without modification to recreate the results that you created with the Optimization Tool. You can also edit and modify the file and run it from the Command Window. To export data from the Optimization Tool to a file, select File > Generate Code.
The generated file captures the following: The problem definition, including the solver, information on the function to be minimized, algorithm specification, constraints, and start point The options (using optimset) with the currently selected option value Running the file at the Command Window reproduces your problem results. Although you cannot export your problem results to a generated file, you can save them in a MAT-file that you can use with your generated file, by exporting the results using the Export to Workspace dialog box, then saving the data to a MAT-file from the Command Window.
Optimization Tool Examples
In this section. About Optimization Tool Examples on page 5-21 Optimization Tool with the fmincon Solver on page 5-21 Optimization Tool with the lsqlin Solver on page 5-25
About Optimization Tool Examples
This section contains two examples showing how to use the Optimization Tool to solve representative problems. There are other examples available: Problem Formulation: Rosenbrocks Function on page 1-4 and Example: Constrained Minimization Using fmincons Interior-Point Algorithm with Analytic Hessian on page 6-53 in this Users Guide, and several in the Global Optimization Toolbox Users Guide.
Optimization Tool with the fmincon Solver
This example shows how to use the Optimization Tool with the fmincon solver to minimize a quadratic subject to linear and nonlinear constraints and bounds. Consider the problem of finding [x1, x2] that solves

min f ( x) = x12 + x22

subject to the constraints

Optimization Theory Overview
Optimization techniques are used to find a set of design parameters, x = {x1,x2,.,xn}, that can in some way be defined as optimal. In a simple case this might be the minimization or maximization of some system characteristic that is dependent on x. In a more advanced formulation the objective function, f(x), to be minimized or maximized, might be subject to constraints in the form of equality constraints, Gi(x) = 0 ( i = 1,.,me); inequality constraints, Gi( x) 0 (i = me + 1,.,m); and/or parameter bounds, xl, xu. A General Problem (GP) description is stated as

subject to

Gi ( x) = 0 i = 1,., me , Gi ( x) 0 i = me + 1,., m,
where x is the vector of length n design parameters, f(x) is the objective function, which returns a scalar value, and the vector function G(x) returns a vector of length m containing the values of the equality and inequality constraints evaluated at x. An efficient and accurate solution to this problem depends not only on the size of the problem in terms of the number of constraints and design variables but also on characteristics of the objective function and constraints. When both the objective function and the constraints are linear functions of the design variable, the problem is known as a Linear Programming (LP) problem. Quadratic Programming (QP) concerns the minimization or maximization of a quadratic objective function that is linearly constrained. For both the LP and QP problems, reliable solution procedures are readily available. More difficult to solve is the Nonlinear Programming (NP) problem in which the objective function and constraints can be nonlinear functions of the design variables. A solution of the NP problem generally requires an iterative procedure to establish a direction of search at each major iteration. This is usually achieved by the solution of an LP, a QP, or an unconstrained subproblem.
Unconstrained Nonlinear Optimization Algorithms
In this section. Definition on page 6-3 Large Scale fminunc Algorithm on page 6-3 Medium Scale fminunc Algorithm on page 6-6 fminsearch Algorithm on page 6-12

Definition

Unconstrained minimization is the problem of finding a vector x that is a local minimum to a scalar function f(x):
The term unconstrained means that no restriction is placed on the range of x.
Large Scale fminunc Algorithm
Trust-Region Methods for Nonlinear Minimization
Many of the methods used in Optimization Toolbox solvers are based on trust regions, a simple yet powerful concept in optimization. To understand the trust-region approach to optimization, consider the unconstrained minimization problem, minimize f(x), where the function takes vector arguments and returns scalars. Suppose you are at a point x in n-space and you want to improve, i.e., move to a point with a lower function value. The basic idea is to approximate f with a simpler function q, which reasonably reflects the behavior of function f in a neighborhood N around the point x. This neighborhood is the trust region. A trial step s is computed by minimizing (or approximately minimizing) over N. This is the trust-region subproblem,

x(n+1) cc m c r v(n+1) x(1)
Unconstrained Nonlinear Optimization Examples
In this section. Example: fminunc Unconstrained Minimization on page 6-15 Example: Nonlinear Minimization with Gradient and Hessian on page 6-17 Example: Nonlinear Minimization with Gradient and Hessian Sparsity Pattern on page 6-18
Example: fminunc Unconstrained Minimization
Consider the problem of finding a set of values [x1, x2] that solves
min f ( x) = e xx1 + 2 x2 + 4 x1 x2 + 2 x2 + 1. x

(6-16)

To solve this two-dimensional problem, write a file that returns the function value. Then, invoke the unconstrained minimization routine fminunc.
Step 1: Write a file objfun.m.
This code ships with the toolbox. To view, enter type objfun:
function f = objfun(x) f = exp(x(1)) * (4*x(1)^2 + 2*x(2)^2 + 4*x(1)*x(2) + 2*x(2) + 1);
Step 2: Invoke one of the unconstrained optimization routines.
x0 = [-1,1]; % Starting guess options = optimset('LargeScale','off'); [x,fval,exitflag,output] = fminunc(@objfun,x0,options);
This produces the following output:

View the results:

x,fval,exitflag,output x = 0.5000 fval = 3.6609e-015 exitflag = 1 output = iterations: funcCount: stepsize: firstorderopt: algorithm: message: -1.0000
1 1.2284e-007 'medium-scale: Quasi-Newton line search' [1x438 char]
The exitflag tells whether the algorithm converged. exitflag = 1 means a local minimum was found. The meanings of exitflags are given in function reference pages. The output structure gives more details about the optimization. For fminunc, it includes the number of iterations in iterations, the number of function evaluations in funcCount, the final step-size in stepsize, a measure of first-order optimality (which in this unconstrained case is the infinity norm of the gradient at the solution) in firstorderopt, the type of algorithm used in algorithm, and the exit message (the reason the algorithm stopped). Pass the variable options to fminunc to change characteristics of the optimization algorithm, as in
x = fminunc(@objfun,x0,options); options is a structure that contains values for termination tolerances and algorithm choices. Create an options structure using the optimset function: options = optimset('LargeScale','off');
You can also create an options structure by exporting from the Optimization Tool. In this example, we have turned off the default selection of the large-scale algorithm and so the medium-scale algorithm is used. Other options include controlling the amount of command line display during the optimization iteration, the tolerances for the termination criteria, whether a user-supplied gradient or Jacobian is to be used, and the maximum number of iterations or function evaluations. See optimset, the individual optimization functions, and Optimization Options Reference on page 9-7 for more options and information.

To allow efficient computation of the sparse finite-difference approximation of the Hessian matrix H(x), the sparsity structure of H must be predetermined. In this case assume this structure, Hstr, a sparse matrix, is available in file brownhstr.mat. Using the spy command you can see that Hstr is indeed sparse (only 2998 nonzeros). Use optimset to set the HessPattern option to Hstr. When a problem as large as this has obvious sparsity structure, not setting the HessPattern option requires a huge amount of unnecessary memory and computation because fminunc attempts to use finite differencing on a full Hessian matrix of one million nonzero entries. You must also set the GradObj option to 'on' using optimset, since the gradient is computed in brownfg.m. Then execute fminunc as shown in Step 2.
fun = @brownfg; load brownhstr % Get Hstr, structure of the Hessian spy(Hstr) % View the sparsity structure of Hstr n = 1000; xstart = -ones(n,1); xstart(2:2:n,1) = 1; options = optimset('GradObj','on','HessPattern',Hstr); [x,fval,exitflag,output] = fminunc(fun,xstart,options);
This 1000-variable problem is solved in seven iterations and seven conjugate gradient iterations with a positive exitflag indicating convergence. The final function value and measure of optimality at the solution x are both close to zero (for fminunc, the first-order optimality is the infinity norm of the gradient of the function, which is zero at a local minimum):
exitflag,fval,output exitflag = 1 fval = 7.4738e-017 output = iterations: funcCount: cgiterations: firstorderopt: algorithm: message: constrviolation: 7 7.9822e-010 'large-scale: trust-region Newton' [1x498 char] []
Constrained Nonlinear Optimization Algorithms
In this section. Definition on page 6-21 fmincon Trust Region Reflective Algorithm on page 6-21 fmincon Active Set Algorithm on page 6-27 fmincon SQP Algorithm on page 6-37 fmincon Interior Point Algorithm on page 6-38 fminbnd Algorithm on page 6-42 fseminf Problem Formulation and Algorithm on page 6-42
Constrained minimization is the problem of finding a vector x that is a local minimum to a scalar function f(x) subject to constraints on the allowable x:
such that one or more of the following holds: c(x) 0, ceq(x) = 0, Ax b, Aeqx = beq, l x u. There are even more constraints used in semi-infinite programming; see fseminf Problem Formulation and Algorithm on page 6-42.
fmincon Trust Region Reflective Algorithm

that returns the value at both constraints at the current x in a vector c. The constrained optimizer, fmincon, is then invoked. Because fmincon expects the constraints to be written in the form c(x) 0, you must rewrite your constraints in the form x1x2 x1 x2 + 1.5 0, x1x0. (6-58)
Step 1: Write a file objfun.m for the objective function.
function f = objfun(x) f = exp(x(1))*(4*x(1)^2 + 2*x(2)^2 + 4*x(1)*x(2) + 2*x(2) + 1);
Step 2: Write a file confun.m for the constraints.
function [c, ceq] = confun(x) % Nonlinear inequality constraints c = [1.5 + x(1)*x(2) - x(1) - x(2); -x(1)*x(2) - 10]; % Nonlinear equality constraints ceq = [];
Step 3: Invoke constrained optimization routine.
x0 = [-1,1]; % Make a starting guess at the solution options = optimset('Algorithm','active-set'); [x,fval] =. fmincon(@objfun,x0,[],[],[],[],[],[],@confun,options); fmincon produces the solution x with function value fval: x,fval x = -9.5474 fval = 0.0236

1.0474

You can evaluate the constraints at the solution by entering

[c,ceq] = confun(x)

This returns numbers close to zero, such as
c = 1.0e-007 * -0.9032 0.9032 ceq = []
Note that both constraint values are, to within a small tolerance, less than or equal to 0; that is, x satisfies c(x) 0.
Example: Bound Constraints
The variables in x can be restricted to certain limits by specifying simple bound constraints to the constrained optimizer function. For fmincon, the command
x = fmincon(@objfun,x0,[],[],[],[],lb,ub,@confun,options);
limits x to be within the range lb
To restrict x in Equation 6-57 to be greater than 0 (i.e., x1 0, x1 0), use the commands
x0 = [-1,1]; % Make a starting guess at the solution lb = [0,0]; % Set lower bounds ub = [ ]; % No upper bounds options = optimset('Algorithm','active-set'); [x,fval] =. fmincon(@objfun,x0,[],[],[],[],lb,ub,@confun,options);
Note that to pass in the lower bounds as the seventh argument to fmincon, you must specify values for the third through sixth arguments. In this example, we specified [] for these arguments since there are no linear inequalities or linear equalities. fmincon produces the following results:

Figure 6-4:

Levenberg-Marquardt Method on Rosenbrocks Function

Gauss-Newton Method

The line search procedures used in conjunction with a quasi-Newton method are used as part of the nonlinear least-squares (LS) optimization routines,
lsqnonlin and lsqcurvefit. In the least-squares problem a function f(x) is minimized that is a sum of squares.

min f ( x) = F ( x)

= Fi2 ( x).

(6-107)

Problems of this type occur in a large number of practical applications, especially when fitting model functions to data, i.e., nonlinear parameter estimation. They are also prevalent in control where you want the output, y(x,t), to follow some continuous model trajectory, (t), for vector x and scalar t. This problem can be expressed as
min ( y( x, t) (t) ) dt, n

(6-108)

where y(x,t) and (t) are scalar functions. When the integral is discretized using a suitable quadrature formula, the above can be formulated as a least-squares problem:
min f ( x) = ( y( x, ti ) (ti ) ) , n

(6-109)

where y and include the weights of the quadrature scheme. Note that in this problem the vector F(x) is
y( x, t1 ) (t1 ) y( x, t ) (t ) . F ( x) = . y( x, tm ) (tm )
In problems of this kind, the residual F(x) is likely to be small at the optimum since it is general practice to set realistically achievable target trajectories. Although the function in LS can be minimized using a general unconstrained minimization technique, as described in Basics of Unconstrained Optimization on page 6-6, certain characteristics of the
problem can often be exploited to improve the iterative efficiency of the solution procedure. The gradient and Hessian matrix of LS have a special structure. Denoting the m-by-n Jacobian matrix of F(x) as J(x), the gradient vector of f(x) as G(x), the Hessian matrix of f(x) as H(x), and the Hessian matrix of each Fi(x) as Hi(x), you have
G( x) = 2 J ( x)T F ( x) H ( x) = 2 J ( x)T J ( x) + 2Q( x),

This function is referred to as the generalized Rosenbrock function. It consists of n squared terms involving n unknowns. Before you can use fsolve to find the values of x such that F(x) = 0, i.e., obtain the minimum of the generalized Rosenbrock function, you must rewrite the function as the following equivalent system of nonlinear equations:

F (1) = 1 x1

2 F (2) = 10 x2 x1

F (3) = 1 x3

2 F (4) = 10 x4 x3

F (n 1) = 1 xn1

2 F (n) = 10 xn xn1.
This system is square, and you can use fsolve to solve it. As the example demonstrates, this system has a unique solution given by xi = 1, i = 1,.,n.
Step 1: Write a file bananaobj.m to compute the objective function values and the Jacobian.
function [F,J] = bananaobj(x) % Evaluate the vector function and the Jacobian matrix for % the system of nonlinear equations derived from the general % n-dimensional Rosenbrock function. % Get the problem size n = length(x); if n == 0, error('Input vector, x, is empty.'); end if mod(n,2) ~= 0, error('Input vector, x ,must have an even number of components.'); end
% Evaluate the vector function odds = 1:2:n;
evens = 2:2:n; F = zeros(n,1); F(odds,1) = 1-x(odds);
F(evens,1) = 10.*(x(evens)-x(odds).^2); % Evaluate the Jacobian matrix if nargout > 1 if nargout > 1 c = -ones(n/2,1); d = 10*ones(n/2,1); e = -20.*x(odds); J = C + D + E; end C = sparse(odds,odds,c,n,n); D = sparse(evens,evens,d,n,n); E = sparse(evens,odds,e,n,n);
Step 2: Call the solve routine for the system of equations.
n = 64; x0(1:n,1) = -1.9; x0(2:2:n,1) = 2; options=optimset('Display','iter','Jacobian','on'); [x,F,exitflag,output,JAC] = fsolve(@bananaobj,x0,options);
Use the starting point x(i) = 1.9 for the odd indices, and x(i) = 2 for the even indices. Set Display to 'iter' to see the solvers progress. Set Jacobian to 'on' to use the Jacobian defined in bananaobj.m. The fsolve function generates the following output:
Norm of Iteration 10 Func-count 11 f(x) 8563.84 3093.71 225.104 212.48 212.48 102.771 102.771 87.7443 74.1426 74.1426 52.2.5 6.25 6.25 1.5625 3.90625 0.976563 2.44141 2.44141 0.610352 step First-order optimality 34.8 34.1 34.1 6.39 6.39 2.19 6.27 6.27 1.52 Trust-region radius 2.5 6.25 6.25 1.56 3.91 0.977 2.44 2.44 0.61
41.3297 34.5115 16.9716 8.16797 3.55178 1.38476 0.219553 0
1.52588 1.52588 1.52588 1.52588 1.52588 1.52588 1.16206 0.0468565

When parfor Runs In Parallel
Using Parallel Computing with fmincon, fgoalattain, and fminimax
In this section. Using Parallel Computing with Multicore Processors on page 7-5 Using Parallel Computing with a Multiprocessor Network on page 7-6 Testing Parallel Computations on page 7-7
Using Parallel Computing with Multicore Processors
If you have a multicore processor, you might see speedup using parallel processing. You can establish a matlabpool of several parallel workers with a Parallel Computing Toolbox license. For a description of Parallel Computing Toolbox software, and the maximum number of parallel workers, see Product Overview. Suppose you have a dual-core processor, and want to use parallel computing: Enter

matlabpool open 2

at the command line. The 2 specifies the number of MATLAB processes to start.
For command-line use, enter
options = optimset('UseParallel','always');
For Optimization Tool, check Options > Approximated derivatives > Evaluate in parallel.
When you run an applicable solver with options, applicable solvers automatically use parallel computing.
To stop computing optimizations in parallel, set UseParallel to 'never', or set the Optimization Tool not to compute in parallel. To halt all parallel computation, enter

matlabpool close

Using Parallel Computing with a Multiprocessor Network
If you have multiple processors on a network, use Parallel Computing Toolbox functions and MATLAB Distributed Computing Server software to establish parallel computation. Here are the steps to take:
1 Make sure your system is configured properly for parallel computing.
Check with your systems administrator, or refer to the Parallel Computing Toolbox documentation, or the Administrator Guide documentation for MATLAB Distributed Computing Server. To perform a basic check:
a At the command line, enter

matlabpool open conf

matlabpool open conf n
where conf is your configuration, and n is the number of processors you want to use.
b If network_file_path is the network path to your objective or constraint

functions, enter

pctRunOnAll('addpath network_file_path')
so the worker processors can access your objective or constraint files.
c Check whether a file is on the path of every worker by entering