Constrained Optimization: Learn It 1

Giselle Miller

Constrained Optimization: Learn It 1

Solve optimization problems with one constraint
Solve optimization problems with two constraints

Lagrange Multipliers

Solving optimization problems for functions of two or more variables can be similar to solving such problems in single-variable calculus. However, techniques for dealing with multiple variables allow us to solve more varied optimization problems for which we need to deal with additional conditions or constraints. Let’s look at an example.

Pro-T company has developed a profit model that depends on the number [latex]x[/latex] of golf balls sold per month (measured in thousands), and the number of hours per month of advertising [latex]y[/latex], according to the function:

[latex]z=f(x,y)=48x+96y-x^{2}-2xy-9y^{2}[/latex]

where [latex]z[/latex] is measured in thousands of dollars. The maximum number of golf balls that can be produced and sold is [latex]50,000[/latex], and the maximum number of hours of advertising that can be purchased is [latex]25[/latex]. Our goal is to find the values of [latex]x[/latex] and [latex]y[/latex] that maximize profit.

Now suppose we add a budgetary constraint, such as [latex]20x+4y\leq 216[/latex], that takes into account the cost of producing the golf balls and the number of advertising hours purchased per month. The goal is still to maximize profit, but now there’s a different type of constraint on the values of [latex]x[/latex] and [latex]y[/latex].

This constraint, combined with the profit function, creates what’s called an optimization problem. The profit function [latex]f(x, y)[/latex] is the objective function—the function we want to maximize (or minimize in other contexts). The budgetary constraint [latex]20x+4y\leq 216[/latex] is the constraint function—it limits the possible values of [latex]x[/latex] and [latex]y[/latex] we can consider.

Here’s a graph of various level curves of the function [latex]f(x, y)[/latex].

A series of rotated ellipses that become increasingly large. The smallest one is marked f(x, y) = 400, and the biggest one is marked f(x, y) = 150. — Figure 1. Graph of level curves of the function [latex]\small{f(x, y)=48x+96y-x^{2}-2xy-9y^{2}}[/latex] corresponding to [latex]\small{c=150,250,350, \text{ and } 400}[/latex].

In Figure 1, the value [latex]c[/latex] represents different profit levels (values of the function [latex]f[/latex]). As [latex]c[/latex] increases, the curve shifts to the right. Since our goal is to maximize profit, we want to choose a curve as far to the right as possible. If there was no restriction on production or advertising, we could produce as many golf balls as we want and advertise as much as we want—there would be no maximum profit.

Unfortunately, we have a budgetary constraint modeled by [latex]20x+4y\leq 216[/latex]. Figure 2 shows how this constraint interacts with the profit function by superimposing the line [latex]20x+4y=216[/latex] on the previous graph.

A series of rotated ellipses that become increasingly large. On the smallest ellipse, which is red, there is a tangent line marked with equation 20x + 4y = 216 that appears to touch the ellipse near (10, 4). — Figure 2. Graph of level curves of the function [latex]\small{f(x, y)=48x+96y-x^{2}-2xy-9y^{2}}[/latex] corresponding to [latex]\small{c=150,250,350, \text{ and } 395}[/latex]. The red graph is the constraint function.

The maximum profit occurs when the level curve is as far to the right as possible. However, the production level corresponding to this maximum must also satisfy the budgetary constraint, so the point must lie on (or to the left of) the red line in Figure 2. Inspection reveals that this point exists where the line is tangent to the level curve of [latex]f[/latex]. Trial and error suggests this profit level is around [latex]395[/latex], when [latex]x[/latex] and [latex]y[/latex] are both just less than [latex]5[/latex].

From a theoretical standpoint, at the point where the profit curve is tangent to the constraint line, the gradient of both functions evaluated at that point must point in the same (or opposite) direction. Recall that the gradient of a function of more than one variable is a vector. If two vectors point in the same (or opposite) directions, then one must be a constant multiple of the other. This idea is the basis of the method of Lagrange multipliers.

method of Lagrange multipliers: one constraint

Let [latex]f[/latex] and [latex]g[/latex] be functions of two variables with continuous partial derivatives at every point of some open set containing the smooth curve [latex]g(x,y)=0[/latex]. Suppose that [latex]f[/latex], when restricted to points on the curve [latex]g(x,y)=0[/latex], has a local extremum at the point [latex](x_0,y_0)[/latex] and that [latex]\nabla{g}(x_0,y_0)\ne0[/latex]. Then there is a number [latex]\lambda[/latex] called a Lagrange multiplier, for which:

[latex]\nabla{f}(x_0,y_0)=\lambda\nabla{g}(x_0,y_0)[/latex]

Proof

Assume that a constrained extremum occurs at the point [latex](x_0, y_0)[/latex]. Furthermore, we assume that the equation [latex]g(x, y)=0[/latex] can be smoothly parameterized as

[latex]x=x(s)[/latex] and [latex]y=y(s)[/latex]

where s is an arc length parameter with reference point [latex](x_0, y_0)[/latex] at [latex]s=0[/latex]. Therefore, the quantity [latex]z=f(x(s), y(s))[/latex] has a relative maximum or relative minimum at [latex]s=0[/latex], and this implies that [latex]\frac{dz}{ds}=0[/latex] at that point. From the chain rule,

[latex]\large{\frac{dz}{ds}=\frac{\partial f}{\partial x}\cdot\frac{\partial x}{\partial s}+\frac{\partial f}{\partial y}\cdot\frac{\partial y}{\partial s}=\left(\frac{\partial f}{\partial x}{\bf{\hat{i}}}+\frac{\partial f}{\partial y}{\bf{\hat{j}}}\right)\cdot\left(\frac{\partial x}{\partial s}{\bf{\hat{i}}}+\frac{\partial y}{\partial s}{\bf{\hat{j}}}\right)=0,}[/latex]

where the derivatives are all evaluated at [latex]s=0[/latex]. However, the first factor in the dot product is the gradient of [latex]f[/latex], and the second factor is the unit tangent vector [latex]\text{T}(0)[/latex] to the constraint curve. Since the point [latex](x_0, y_0)[/latex] corresponds to [latex]s=0[/latex], it follows from this equation that

[latex]\large{\nabla{f}(x_0,y_0)\cdot{\text{T}}(0)=0},[/latex]

which implies that the gradient is either [latex]0[/latex] or is normal to the constraint curve at a constrained relative extremum. However, the constraint curve [latex]g(x, y)=0[/latex] is a level curve for the function [latex]g(x, y)[/latex] so that if [latex]\nabla{g}(x_0,y_0)\ne0[/latex] then [latex]\nabla{g}(x_0,y_0)[/latex] is normal to this curve at [latex](x_0, y_0)[/latex] It follows, then, that there is some scalar [latex]\lambda[/latex] such that

[latex]\large{\nabla{f}(x_0,y_0)=\lambda\nabla{g}(x_0,y_0)}[/latex]

[latex]_\blacksquare[/latex]

To apply the method of Lagrange multipliers to optimization problems, we need a systematic strategy.

Steps for Using Lagrange Multipliers

Identify the objective and constraint functions. Determine the objective function [latex]f(x, y)[/latex] and the constraint function [latex]g(x, y)[/latex]. Does the problem ask you to maximize or minimize?
Set up the system of equations:

[latex]\begin{array}\nabla{f}(x_0,y_0)&=\lambda\nabla{g}(x_0,y_0) \\ g(x_0,y_0)&=0 \end{array}[/latex]

Solve the system for [latex]x_0[/latex] and [latex]y_0[/latex].
Evaluate and compare. The largest of the values of [latex]f[/latex] at the solutions found in step 3 maximizes [latex]f[/latex]; the smallest minimizes [latex]f[/latex].

Use the method of Lagrange multipliers to find the minimum value of [latex]f(x,y)=x^2+4y^2-2x+8y[/latex] subject to the constraint [latex]x+2y=7[/latex].

Let’s follow the problem-solving strategy:

The optimization function is [latex]f(x,y)=x^2+4y^2-2x+8y[/latex]. To determine the constraint function, we must first subtract [latex]7[/latex] from both sides of the constraint. This gives [latex]x+2y-7=0[/latex]. The constraint function is equal to the left-hand side, so [latex]g(x,y)=x+2y-7[/latex]. The problem asks us to solve for the minimum value of [latex]f[/latex], subject to the constraint (see the following graph).

Two rotated ellipses, one within the other. On the largest ellipse, which is marked f(x, y) = 26, there is a tangent line marked with equation x + 2y = 7 that appears to touch the ellipse near (5, 1). — Figure 3. Graph of level curves of the function [latex]\small{f(x, y)=x^{2}+4y^{2}-2x+8y}[/latex] corresponding to [latex]\small{c=10\text{ and } 26}[/latex]. The red graph is the constraint function.

2. We then must calculate the gradients of both [latex]f[/latex] and [latex]g[/latex]:
[latex]\hspace{8cm}\begin{align} \nabla{f}(x,y)&=(2x-2){\bf{i}}+(8y+8){\bf{j}} \\ \nabla{g}(x,y)&={\bf{i}}+2{\bf{j}}. \end{align}[/latex]
The equation [latex]\nabla{f}(x_0,y_0)=\lambda\nabla{g}(x_0,y_0)[/latex] becomes

[latex](2x_0-2){\bf{i}}+(8y_0+8){\bf{j}}=\lambda({\bf{i}}+2{\bf{j}}),[/latex]

which can be rewritten as

[latex](2x_0-2){\bf{i}}+(8y_0+8){\bf{j}}=\lambda{\bf{i}}+2\lambda{\bf{j}}.[/latex]

Next, we set the coefficients of [latex]\bf{i}[/latex] and [latex]\bf{j}[/latex] equal to each other:

[latex]\hspace{10cm}\begin{align} 2x_0-2&=\lambda \\ 8y_0+8&=2\lambda. \end{align}[/latex]

The equation [latex]g(x_0,y_0)=0[/latex] becomes [latex]x_0+2y_0-7=0[/latex]. Therefore, the system of equations that needs to be solved is

[latex]\hspace{9cm}\begin{align} 2x_0-2&=\lambda \\ 8y_0+8&=2\lambda \\ x_0+2y_0-7&=0. \end{align}[/latex]

3. This is a linear system of three equations in three variables. We start by solving the second equation for [latex]\lambda[/latex] and substituting it into the first equation. This gives [latex]\lambda=4y_0+4[/latex], so substituting this into the first equation gives

[latex]2x_0-2=4y_0+4[/latex].

Solving this equation for [latex]x_0[/latex] gives [latex]x_0=2y_0+3[/latex]. We then substitute this into the third equation:
[latex]\hspace{9cm}\begin{align} (2y_0+3)+2y_0-7&=0 \\ 4y_0-4&=0 \\ y_0=1. \end{align}[/latex]

Since [latex]x_0=2y_0+3[/latex], this gives [latex]x_0=5[/latex].

4. Next, we substitute [latex](5,1)[/latex] into [latex]f(x,y)=x^2+4y^2-2x+8y[/latex], gives [latex]f(5,1)=5^2+4(1)^2-2(5)+8(1)=27[/latex]. To ensure this corresponds to a minimum value on the constraint function, let’s try some other values, such as the intercepts of [latex]g(x,y)=0[/latex], Which are [latex](7,0)[/latex] and [latex](0,3.5)[/latex]. We get [latex]f(7,0)=35[/latex] and [latex]f(0.3.5)=77[/latex], so it appears [latex]f[/latex] has a minimum at [latex](5,1)[/latex].

Return to the golf ball manufacturer problem. The golf ball manufacturer, Pro-T, has developed a profit model that depends on the number [latex]x[/latex] of golf balls sold per month (measured in thousands), and the number of hours per month of advertising [latex]y[/latex], according to the function

[latex]z=f(x, y)=48x+96y-x^{2}-2xy-9y^{2}[/latex],

where [latex]z[/latex] is measured in thousands of dollars. The budgetary constraint function relating the cost of the production of thousands golf balls and advertising units is given by [latex]20x+4y=216[/latex]. Find the values of [latex]x[/latex] and [latex]y[/latex] that maximize profit, and find the maximum profit.

Again, we follow the problem-solving strategy:

The optimization function is [latex]f(x, y)=48x+96y-x^{2}-2xy-9y^{2}[/latex]. To determine the constraint function, we first subtract 216 from both sides of the constraint, then divide both sides by 4, which gives [latex]5x+y-54=0[/latex]. The constraint function is equal to the left-hand side, so [latex]g(x, y)=5x+y-54[/latex]. The problem asks us to solve for the maximum value of [latex]f[/latex] subject to this constraint.
So, we calculate the gradients of both [latex]f[/latex] and [latex]g[/latex]:
[latex]\hspace{6cm}\begin{align} \nabla{f}(x,y)&=(48-2x-2y){\bf{i}}+(96-2x-18y){\bf{j}} \\ \nabla{g}(x,y)&=5{\bf{i}}+{\bf{j}}. \end{align}[/latex]
The equation [latex]\nabla{f}(x_0,y_0)=\lambda\nabla{g}(x_0,y_0)[/latex] becomes

[latex]\hspace{6cm}(48-2x_0-2y_0){\bf{i}}+(96-2x_0-18y_0){\bf{j}}=\lambda(5{\bf{i}}+{\bf{j}})[/latex],
which can be rewritten as

[latex](48-2x_0-2y_0){\bf{i}}+(96-2x_0-18y_0){\bf{j}}=\lambda5{\bf{i}}+\lambda{\bf{j}}[/latex].

We then set the coefficients of [latex]\bf{i}[/latex] and [latex]\bf{j}[/latex] equal to each other:

[latex]\hspace{8cm}\begin{align} 48-2x_0-2y_0&=5\lambda \\ 96-2x_0-18y_0&=\lambda. \end{align}[/latex]

The equation [latex]g(x_0, y_0)=0[/latex] becomes [latex]5x_0+y_0-54=0[/latex]. Therefore, the system of equations that needs to be solved is

[latex]\hspace{8cm}\begin{align} 48-2x_0-2y_0&=5\lambda \\ 96-2x_0-18y_0&=\lambda \\ 5x_0+y_0-54&=0. \end{align}[/latex]
We use the left-hand side of the second equation to replace [latex]\lambda[/latex] in the first equation:

[latex]\hspace{8cm}\begin{align} 48-2x_0-2y_0&=5(96-2x_0-18y_0) \\ 48-2x_0-2y_0&=480-10x_0-90y_0 \\ 8x_0&=432-88y_0 \\ x_0&=54-11y_0. \end{align}[/latex]

Then we substitute this into the third equation:

[latex]\hspace{8cm}\begin{align} 5(54-11y_0)+y+0&=0 \\ 270-55y_0+y_0&=0 \\ 216-54y_0&=0 \\ y_0&=4. \end{align}[/latex]

Since [latex]x_0=54-11y_0[/latex], this gives [latex]x_0=10[/latex].
We then substitute [latex](10, 4)[/latex] into [latex]f(x, y)=48x+96y-x^{2}-2xy-9y^{2}[/latex], which gives

[latex]\hspace{5cm}\begin{align} f(10,4)&=48(10)+96(4)-(10)^2-2(10)(4)-9(4)^2 \\ &=480+384-100-80-144=540. \end{align}[/latex]

Therefore the maximum profit that can be attained, subject to budgetary constraints, is [latex]$540,000[/latex] with a production level of [latex]10,000[/latex] golf balls and [latex]4[/latex] hours of advertising bought per month. Let’s check to make sure this truly is a maximum. The endpoints of the line that defines the constraint are [latex](10.8, 0)[/latex] and [latex](0, 54)[/latex]. Let’s evaluate [latex]f[/latex] at both of these points:

[latex]\hspace{5cm}\begin{align} f(10.8,0)&=48(10.8)+96(0)-10.8^2-2(10.8)(0)-9(0)^2=401.76 \\ f(0,54)&=48(0)+96(54)-0^2-2(0)(54)-9(54)^2=-21,060. \end{align}[/latex]

The second value represents a loss, since no golf balls are produced. Neither of these values exceed [latex]540[/latex], so it seems that our extremum is a maximum value of [latex]f[/latex].

Extension to Three Variables

The method of Lagrange multipliers extends naturally to optimization functions with three variables and a single constraint. For example, consider a Cobb-Douglas function [latex]f(x,y,z)=x^{0.2}y^{0.4}z^{0.4}[/latex], where [latex]x[/latex] represents the cost of labor, [latex]y[/latex] represents capital input, and [latex]z[/latex] represents the cost of advertising. The equations to solve are:

[latex]\begin{array}\nabla{f}(x,y,z)&=\lambda\nabla{g}(x,y,z) \\g(x,y,z)&=0\end{array}[/latex]

Find the minimum of the function [latex]f(x, y, z)=x^{2}+y^{2}+z^{2}[/latex] subject to the constraint [latex]x+y+z=1[/latex].