Optimization and Applications: Cheat Sheet

Essential Concepts

Critical Points and Local Extrema

  • A critical point of the function [latex]f(x,y)[/latex] is any point [latex](x_{0},y_{0})[/latex] where either [latex]f_{x}(x_{0},y_{0})=f_{y}(x_{0},y_{0})=0[/latex], or at least one of [latex]f_{x}(x_{0},y_{0})[/latex] and [latex]f_{y}(x_{0},y_{0})[/latex] do not exist.
  • A saddle point is a point [latex](x_{0},y_{0})[/latex] where [latex]f_{x}(x_{0},y_{0})=f_{y}(x_{0},y_{0})=0[/latex], but [latex](x_{0},y_{0})[/latex] is neither a maximum nor a minimum at that point.
  • To find extrema of functions of two variables, first find the critical points, then calculate the discriminant and apply the second derivative test.

Constrained Optimization

  • An objective function combined with one or more constraints is an example of an optimization problem.
  • To solve optimization problems, we apply the method of Lagrange multipliers using a four-step problem-solving strategy.

Key Equations

  • Discriminant
    [latex]D=f_{xx}(x_{0},y_{0})f_{yy}(x_{0},y_{0})-\left(f_{xy}(x_{0},y_{0})\right)^{2}[/latex]
  • Method of Lagrange multipliers, one constraint
    [latex]\begin{array}{cc} \nabla f(x_{0},y_{0}) &= \lambda\nabla g(x_{0},y_{0}) \\ \hfill g(x_{0},y_{0}) &= 0 \hfill \end{array}[/latex]
  • Method of Lagrange multipliers, two constraints
    [latex]\begin{array}{cc} \nabla f(x_{0},y_{0},z_{0}) &= \lambda_{1}\nabla g(x_{0},y_{0},z_{0})+\lambda_{2}\nabla h(x_{0},y_{0},z_{0}) \\ \hfill g(x_{0},y_{0},z_{0}) &= 0 \hfill\\ \hfill h(x_{0},y_{0},z_{0}) &= 0 \hfill \end{array}[/latex]

Glossary

constraint
an inequality or equation involving one or more variables that is used in an optimization problem; the constraint enforces a limit on the possible solutions for the problem
critical point of a function of two variables
the point [latex](x_{0},y_{0})[/latex] is called a critical point of [latex]f(x,y)[/latex] if one of the two following conditions holds:

  1. [latex]f_{x}(x_{0},y_{0})=f_{y}(x_{0},y_{0})=0[/latex]
  2. At least one of [latex]f_{x}(x_{0},y_{0})[/latex] and [latex]f_{y}(x_{0},y_{0})[/latex] do not exist
discriminant
the discriminant of the function [latex]f(x,y)[/latex] is given by the formula [latex]D=f_{xx}(x_{0},y_{0})f_{yy}(x_{0},y_{0})-\left(f_{xy}(x_{0},y_{0})\right)^{2}[/latex]
Lagrange Multiplier
the constant (or constants) used in the method of Lagrange multipliers; in the case of one constant, it is represented by the variable [latex]\lambda[/latex]
method of Lagrange multipliers
a method of solving an optimization problem subject to one or more constraints
objective function
the function that is to be maximized or minimized in an optimization problem
optimization problem
calculation of a maximum or minimum value of a function of several variables, often using Lagrange multipliers
saddle point
given the function [latex]z=f(x,y)[/latex] the point [latex](x_{0},y_{0},f(x_{0},y_{0}))[/latex] is a saddle point if both [latex]f_{x}(x_{0},y_{0})=0[/latex] and [latex]f_{y}(x_{0},y_{0})=0[/latex], but [latex]f[/latex] does not have a local extremum at [latex](x_{0},y_{0})[/latex]