Constrained Optimization: Fresh Take

  • Solve optimization problems with one constraint
  • Solve optimization problems with two constraints

Lagrange Multipliers

The Main Idea 

When you need to optimize a function subject to a constraint, Lagrange multipliers provide an elegant alternative to parameterizing boundaries. The key insight: at an optimum point, the gradient of your objective function must be parallel to the gradient of your constraint.

To maximize or minimize [latex]f(x, y)[/latex] subject to constraint [latex]g(x, y) = 0[/latex], look for points where:

[latex]\nabla f(x_0, y_0) = \lambda \nabla g(x_0, y_0)[/latex]

The scalar [latex]\lambda[/latex] (lambda) is called the Lagrange multiplier. Geometrically, this says the level curves of [latex]f[/latex] and [latex]g[/latex] are tangent at the optimal point.

Problem-Solving Strategy

  1. Identify your functions: Determine the objective function [latex]f[/latex] (what you’re optimizing) and constraint [latex]g[/latex] (rewritten so it equals zero)
  2. Set up the system: Write [latex]\nabla f = \lambda \nabla g[/latex] and [latex]g = 0[/latex], which gives you three equations in three unknowns
  3. Solve the system: Find [latex]x_0[/latex], [latex]y_0[/latex], and [latex]\lambda[/latex]
  4. Evaluate and compare: Calculate [latex]f[/latex] at each solution point. Test a few other constraint-satisfying points to verify you found a maximum or minimum

Why This Works

If [latex]f[/latex] has an extremum on the constraint curve [latex]g = 0[/latex], then moving along the constraint shouldn’t change [latex]f[/latex] to first order. This happens when [latex]\nabla f[/latex] is perpendicular to the constraint curve—which means [latex]\nabla f[/latex] must be parallel to [latex]\nabla g[/latex] (since [latex]\nabla g[/latex] is also perpendicular to the curve).

Extension to Three Variables

For [latex]f(x, y, z)[/latex] with constraint [latex]g(x, y, z) = 0[/latex], the method is identical: solve [latex]\nabla f = \lambda \nabla g[/latex] and [latex]g = 0[/latex], giving four equations in four unknowns.

Use the method of Lagrange multipliers to find the maximum value of [latex]f(x, y)=9x^{2}+36xy-4y^{2}-18x-8y[/latex] subject to the constraint [latex]3x+4y=32[/latex].

A company has determined that its production level is given by the Cobb-Douglas function [latex]f(x,y)=2.5x^{0.45}y^{0.55}[/latex] where [latex]x[/latex] represents the total number of labor hours in [latex]1[/latex] year and [latex]y[/latex] represents the total capital input for the company. Suppose [latex]1[/latex] unit of labor costs [latex]$40[/latex] and [latex]1[/latex] unit of capital costs [latex]$50[/latex]. Use the method of Lagrange multipliers to find the maximum value of [latex]f(x,y)=2.5x^{0.45}y^{0.55}[/latex] subject to a budgetary constraint of [latex]$500,000[/latex] per year.

Watch the following video to see the worked solution to the above example.

Use the method of Lagrange multipliers to find the minimum value of the function

[latex]\large{f(x,y,z)=x+y+z}[/latex]

subject to the constraint [latex]x^2+y^2+z^2=1[/latex]. 

Problems with Two Constraints

The Main Idea 

When optimizing a function of three variables subject to two constraints, you need two Lagrange multipliers. The method extends naturally from the single-constraint case, but the algebra becomes more involved.

To optimize [latex]f(x, y, z)[/latex] subject to two constraints [latex]g(x, y, z) = 0[/latex] and [latex]h(x, y, z) = 0[/latex], solve:

[latex]\nabla f(x_0, y_0, z_0) = \lambda_1 \nabla g(x_0, y_0, z_0) + \lambda_2 \nabla h(x_0, y_0, z_0)[/latex]

along with [latex]g(x_0, y_0, z_0) = 0[/latex] and [latex]h(x_0, y_0, z_0) = 0[/latex].

What This Means

At an optimal point, the gradient of [latex]f[/latex] must lie in the plane spanned by the gradients of [latex]g[/latex] and [latex]h[/latex]. Geometrically, you’re finding extrema along the curve where two surfaces intersect.

Problem-Solving Strategy

  1. Identify your functions: Write down [latex]f[/latex] (objective), [latex]g[/latex] (first constraint), and [latex]h[/latex] (second constraint), each set equal to zero
  2. Calculate all three gradients: [latex]\nabla f[/latex], [latex]\nabla g[/latex], and [latex]\nabla h[/latex]
  3. Set up the system: Write [latex]\nabla f = \lambda_1 \nabla g + \lambda_2 \nabla h[/latex], [latex]g = 0[/latex], and [latex]h = 0[/latex]—this gives five equations in five unknowns
  4. Solve systematically: Use substitution and elimination to reduce variables. Look for relationships between [latex]\lambda_1[/latex] and [latex]\lambda_2[/latex] that simplify the system
  5. Evaluate [latex]f[/latex] at all solution points and compare to find maximum and minimum values

Critical Tip

The algebra gets complex quickly. Work methodically, solve equations in stages, and check your arithmetic carefully. Often you’ll eliminate one multiplier, then the other, before solving for [latex]x_0[/latex], [latex]y_0[/latex], and [latex]z_0[/latex].

Use the method of Lagrange multipliers to find the minimum value of the function

[latex]\large{f(x,y,z)=x^2+y^2+z^2}[/latex]

subject to the constraints [latex]2x+y+2z=9[/latex] and [latex]5x+5y+7z=29[/latex]. 

Watch the following video to see the worked solution to the above example.