EMGT 6412/MATH 6665
Mathematical Programming
Spring 2016
Mathematical Programming:
Introduction to Linear
Programming
Dincer Konur
Engineering Management and Systems
Engineering
1
Outline
• Introduction to Mathematical Programming
• Linear Programming Formulation
– Steps of Formulation
– Examples
– Assumptions
• Geometric Interpretation
– Geometry of LPs
– Properties of LPs
• Linear Programming Format
– Matrix Notation
– Standard Format
Chapter 1
2
Outline
• Introduction to Mathematical Programming
• Linear Programming Formulation
– Steps of Formulation
– Examples
– Assumptions
• Geometric Interpretation
– Geometry of LPs
– Properties of LPs
• Linear Programming Format
– Matrix Notation
– Standard Format
3
Mathematical Programming
• Mathematical programming is selecting the best
option(s) from a set of alternatives
mathematically
– Minimizing/maximizing a function where your
alternatives are defined by functions
• Many industries use mathematical programming
–
–
–
–
Supply chain, logistics, and transportation
Health industry, energy industry, finance, airlines
Manufacturing industry, agriculture industry
Education, Military
• Operations Research raised with WWII
4
Linear Programming
• Linear Programming problem (LP) is a
mathematical programming problem where all of
your functions are linear
– A function is linear when
• The variables have power of 1 and the variables are not
𝑓 𝑥
multiplied with each other
–
–
–
–
𝑓
𝑓
𝑓
𝑓
𝑥 = 3𝑥 + 5  linear
𝑥 = 3𝑥 2 + 5  not linear
𝑥1 , 𝑥2 = 3𝑥1 + 5𝑥2 + 7  linear
𝑥1 , 𝑥2 = 3𝑥1 𝑥2 + 7  not linear
𝑥
5
Outline
• Introduction to Mathematical Programming
• Linear Programming Formulation
– Steps of Formulation
– Examples
– Assumptions
• Geometric Interpretation
– Geometry of LPs
– Properties of LPs
• Linear Programming Format
– Matrix Notation
– Standard Format
6
Linear Programming Formulation
• There are 4 main steps for formulating a
mathematical programming model:
– Step 0: Understand the problem and problem data
• Objective function (cost) coefficients, technological coefficients
– Step 1: Identify the decision variables
• Things we control, and want to solve for
– Step 2: Construct the objective function
• Measure of performance (criterion function)
– Step 3: Construct the constraints
• Restrictions
7
Linear Programming Formulation
• x1,…, xn are the decision variables
• c1, c2,…, cn are the cost coefficients
– c1x1 + c2x2 + … + cnxn is the objective function
• ith constraint:
• Technological
coefficient:
8
Linear Programming Formulation
Deserted Island: Linear programming will save you!
– Suppose that you will be left in a deserted island
– You want to live as many days as you can
– Fortunately (!), you can take a bag with you to the island
The bag can carry at most 50 lbs and there are limited set of items you
can carry
– Bread: You can survive for 2 days with 1 lb of bread
– Steak: You can survive for 5 days with 1 lb of steak (Memphis style
grilled!)
– Chicken: You can survive for 3 days with 1 lb of chicken (southern style
deep fried!)
– Chocolate: You can survive for 6 days with 1 lb of chocolate
• How much of each item to take with you?
9
Linear Programming Formulation
• STEP 0: Know the problem and gather your data
– Your problem is to increase your chance of rescue by maximizing the
number of days you survive
– You can have 1 bag which can carry 50 lbs at most
– You can only put bread, steak, chicken and chocolate into you bag
– Each item enables you survive for a specific number of days for each pound
you take
Bread: 2 days/lb
Steak: 5 days/lb Chicken: 3days/lb Chocolate: 6days/lb
• STEP 1: Identify your decision variables, which are the things you control
– The amount of each item you will take with you
• 𝑥1 : 𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑏𝑟𝑒𝑎𝑑 (𝑙𝑏𝑠)
• 𝑥2 : 𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑠𝑡𝑒𝑎𝑘 (𝑙𝑏𝑠)
• 𝑥3 : 𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑐ℎ𝑖𝑐𝑘𝑒𝑛 (𝑙𝑏𝑠)
• 𝑥4 : 𝑡ℎ𝑒 𝑎𝑚𝑜𝑢𝑛𝑡 𝑜𝑓 𝑐ℎ𝑜𝑐𝑜𝑙𝑎𝑡𝑒 𝑙𝑏𝑠
Warning: Always be careful with the metrics (try to use the same metrics)
10
Linear Programming Formulation
• STEP 2: Define your objective function and objective
– Your objective function is the measure of performance as a result of your
decisions
– Your objective is what you want to do with your objective function
– Recall that you want to maximize the number of days you will survive
Objective
Performance measure, Not a function though
– We know our objective (maximize) and our performance of measure
(number of days). Express the performance of measure as a function of
your decision variables to get the objective function
•
•
•
•
If you take 𝑥1 lbs of bread, you will survive for 2𝑥1 days
If you take 𝑥2 lbs of steak, you will survive for 5𝑥1 days
If you take 𝑥3 lbs of chicken, you will survive for 3𝑥1 days
If you take 𝑥4 lbs of chocolate, you will survive for 6𝑥1 days
– Then objective function in terms of decision variables:
2𝑥1 +5𝑥2 + 3𝑥3 + 6𝑥4
11
Linear Programming Formulation
• STEP 3: Define your restrictions (constraints)
– There may be some restrictions which limit what you can do (hence, they
define set of your alternatives)
– You can carry 1 bag and it can carry 50 lbs at most
𝑥1 + 𝑥2 + 𝑥3 + 𝑥4 ≤ 50
Limit on how much
you can carry
Total amount you
decide to carry
– You cannot get negative amounts!!!
𝑥1 ≥ 0, 𝑥2 ≥ 0, 𝑥3 ≥ 0, 𝑥4 ≥ 0
12
Linear Programming Formulation
• Combine your objective, objective function, and
constraints
𝑀𝑎𝑥𝑖𝑚𝑖𝑧𝑒 2𝑥1 +5𝑥2 + 3𝑥3 + 6𝑥4
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑥1 + 𝑥2 + 𝑥3 + 𝑥4 ≤ 50
𝑥1 ≥ 0, 𝑥2 ≥ 0, 𝑥3 ≥ 0, 𝑥4 ≥ 0
» This is your LP model
» Note that everything is linear!
13
Linear Programming Formulation
• Feed Mix Problem: An agricultural mill manufactures feed for
chickens. This is done by mixing several ingredients, such as corn,
limestone, or alfalfa. The mixing is to be done in such a way that the
feed meets certain levels for different types of nutrients, such as
protein, calcium, carbohydrates, and vitamins.
–
–
–
–
–
Suppose that there are n ingredients j = 1,…, n
Suppose that there are m nutrients i = 1,…, m
Let the unit cost of ingredient j be cj
Suppose that aij is the amount of nutrient i present in a unit of ingredient j
Acceptable lower and upper limits of nutrient i in a unit of the chicken feed
are li’ and ui’, respectively
– Suppose that the mill cannot acquire more than uj units of ingredient j
• What should be the mixing the ingredients such that the cost is
minimized and the restrictions are met?
14
Linear Programming Formulation
• Let the amount of ingredient j to be used be xj
15
Linear Programming Formulation
• Cutting Stock Problem: A manufacturer of metal sheets produces
rolls of standard fixed width w and of standard length l. A large order
is placed by a customer who needs sheets of width w and varying
lengths. The manufacturer would like to cut the standard rolls in such
a way as to satisfy the order and minimize the waste.
– Suppose that bi sheets with length li and width w needed, i = 1,…, m
– There are many ways of cutting the standard sheet in different lengths
– The cutting patterns: j=1,2,…,n
• aij is the number of sheets of
length li in the jth pattern
• Minimize the number of standard rolls used by determining how
many of each pattern cut so that you satisfy the customer
needs?
16
Linear Programming Formulation
• Let xj be the number of standard rolls cut
according to the jth pattern
• This is not a LP
unless we
remove the
integer
constraints!!
• LP models
assume that
decision
variables are
continuous 17
Linear Programming Formulation
• Transportation Problem: The Brazilian coffee company processes
coffee beans into coffee at m plants. The coffee is then shipped every
week to n warehouses in major cities for retail, distribution, and
exporting. Suppose that the unit shipping cost from plant i to
warehouse j is cij. Furthermore, suppose that the production capacity
at plant i is ai and that the demand at warehouse j is bj.
• Practice: Formulate Tanker Scheduling problem
18
Linear Programming Formulation
• More numerical examples in Supplement 1
• Assumptions of Linear Programming:
– Proportionality: Given variable xj, its contribution to cost is cixj and
its contribution to the ith constraint is aijxj
– Additivity: This assumption guarantees that the total cost is the sum
of the individual costs, and that the total contribution to the ith
restriction is the sum of the individual contributions of the individual
activities.
– Divisibility:This assumption ensures that the decision variables can
be divided into any fractional levels so that non-integral values for
the decision variables are permitted.
– Deterministic. The coefficients ci, aij, and bi are all known
deterministically.
19
Outline
• Introduction to Mathematical Programming
• Linear Programming Formulation
– Steps of Formulation
– Examples
– Assumptions
• Geometric Interpretation
– Geometry of LPs
– Properties of LPs
• Linear Programming Format
– Matrix Notation
– Standard Format
20
Geometric Interpretation
• A simple graphical procedure to solve simple LPs
(with two decision variables)
21
Geometric Interpretation
• Illustration:
• Examples in Supplement 1
22
Geometric Interpretation
• There are 4 cases possible for a LP:
– Unique Optimal Solution. If the optimal solution is unique, then it
occurs at an extreme point.
23
Geometric Interpretation
• There are 4 cases possible for a LP:
– Alternative Optimal Solutions. If there are more than one feasible
solutions that are optimal.
24
Geometric Interpretation
• There are 4 cases possible for a LP:
– Unbounded Optimal Objective Value. If there is always a better
solution, we have unboundedness.
• If the feasible region is
bounded, you cannot
have unbounded optimal
objective value
25
Geometric Interpretation
• There are 4 cases possible for a LP:
– Empty Feasible Region. The case when there is no feasible
solution.
26
Outline
• Introduction to Mathematical Programming
• Linear Programming Formulation
– Steps of Formulation
– Examples
– Assumptions
• Geometric Interpretation
– Geometry of LPs
– Properties of LPs
• Linear Programming Format
– Matrix Notation
– Standard Format
27
Linear Programming Format
• Matrix Notation:
– A LP can be stated in a more convenient form using
matrix notation
28
Linear Programming Format
• Standard format:
– Objective is minimization
– All variables are non-negative
– All constraints (except non-negativity) are equality
constraints
– Any linear programming model can be converted to the
standard format!!!
• Convert the variables if needed, convert the constraints if needed and
29
convert the objective function if needed
Linear Programming Format
• Converting to standard format:
– Variables: X1, X2,…, Xn
• Non-negative: 𝑋𝑗 ≥ 0
– Do nothing 
• Non-positive: 𝑋𝑗 ≤ 0
– Then define 𝑌𝑗 = −𝑋𝑗 and replace 𝑋𝑗 with −𝑌𝑗 in the formulation
• Unrestricted: 𝑋𝑗 can be both negative or positive
–
–
–
–
Any number can be written as difference of two positive numbers
Then define 𝑋𝑗 = 𝑋𝑗 ′ − 𝑋𝑗′′ where 𝑋𝑗′ ≥ 0 and 𝑋𝑗′′ ≥ 0
Replace 𝑋𝑗 with 𝑋𝑗 ′ − 𝑋𝑗′′ (one variable is replaced with two)
If 𝑋1, 𝑋2, …, 𝑋𝑘 for some k, then define
» 𝑋𝑗 = 𝑋𝑗 ′ − 𝑋′′ for all j=1,2,…,k where 𝑋𝑗′ ≥ 0 and 𝑋′′ ≥ 0
• Bounded….
30
Linear Programming Format
• Converting to standard format:
– Variables: X1, X2,…, Xn
• Bounded: 𝑋𝑗 ≥ 𝐿𝑗 or 𝑋𝑗 ≤ 𝑈𝑗
–
–
–
–
–
If 𝑋𝑗 ≥ 𝐿𝑗 define 𝑋𝑗 ′ = 𝑋𝑗 − 𝐿𝑗 note that 𝑋𝑗 ′ ≥ 0
Replace 𝑋𝑗 with 𝑋𝑗 ′ + 𝐿𝑗
If 𝑋𝑗 ≤ 𝑈𝑗 define 𝑋𝑗 ′ = 𝑈𝑗 − 𝑋𝑗
Replace 𝑋𝑗 with U𝑗 ′ − 𝑋𝑗′
You can also treat bounded variables as regular <= or >=
constraints and convert them to ‘=‘ constraints
– Constraints:
• Less than or equal to constraints
• Greater than or equal to constraints
Must be
converted to ‘=‘
constraints
31
Linear Programming Format
• Converting to standard format:
– Constraints:
• Greater than or equal to constraint:
– This can be made an ‘=‘ constraint by subtracting a positive slack
variable,
(sometimes referred to si) from left-hand-side
– We will have
and
• Less than or equal to constraint:
– This can be made an ‘=‘ constraint by adding a positive slack
variable,
(sometimes referred to si) to the left-hand-side
– We will have
and
Note: An equality constraint can be transformed into two inequality constraints:
32
Linear Programming Format
• Converting to standard format:
– Objective:
• Example:
Maximize
Objective must be ‘minimize’
-
+
+
Must be ‘=‘
Must be ‘=‘
This is ‘=‘
Convert to >=0
Convert to >=0
33
Linear Programming Format
• Converting to standard format:
– Objective converted!
Objective is ‘minimize’ now
Must be ‘=‘
Must be ‘=‘
This is ‘=‘
Convert to >=0
Convert to >=0
34
Linear Programming Format
• Converting to standard format:
– First constraint: ≤ constraint
•
• Define a slack variable 𝑥4 ≥ 0 and add it to the left side
• −𝑥1 + 3𝑥2 + 𝑥3 + 𝑥4 = 13 and 𝑥4 ≥ 0
−𝒙𝟏 + 𝟑𝒙𝟐 + 𝒙𝟑 + 𝒙𝟒 = 𝟏𝟑
Objective is
‘minimize’
This is ‘=‘
Must be ‘=‘
This is ‘=‘
Convert to >=0
This is ≥0
𝑥4 ≥ 0
Convert to >=0
35
Linear Programming Format
• Converting to standard format:
– Second constraint: ≥ constraint
•
• Define a slack variable 𝑥5 ≥ 0 and subtract it from the left side
• 𝑥1 + 2𝑥2 + 3𝑥3 − 𝑥5 = 12 and 𝑥5 ≥ 0
−𝑥1 + 3𝑥2 + 𝑥3 + 𝑥4 = 13
Objective is
‘minimize’
𝒙𝟏 + 𝟐𝒙𝟐 + 𝟑𝒙𝟑 − 𝒙𝟓 = 𝟏𝟐
This is ‘=‘
This is ‘=‘
This is ‘=‘
Convert to >=0
These are ≥0
𝑥4 ≥ 0
𝑥5 ≥ 0
Convert to >=0
36
Linear Programming Format
• Converting to standard format:
– Objective and constraints are converted, we have:
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜
𝑥1 − 2𝑥2 − 3𝑥3
−𝑥1 + 3𝑥2 + 𝑥3 + 𝑥4
𝑥1 + 2𝑥2 + 3𝑥3
2𝑥1 − 𝑥2 + 𝑥3
Convert to >=0
𝑥1 , 𝑥2 𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑
𝑥3 ≤ −3
𝑥4 ≥ 0
𝑥5 ≥ 0
= 13
− 𝑥5 = 12
=4
37
Linear Programming Format
• Converting to standard format:
– Variables 𝑥1 , 𝑥2 𝑎𝑟𝑒 𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑
• Let 𝑥1 = 𝑥1 ′ − 𝑥1 ′′ where 𝑥1′ ≥ 0, 𝑥1′′ ≥ 0, replace 𝑥1 with 𝑥1 ′ − 𝑥1 ′′
• Let 𝑥2 = 𝑥2 ′ − 𝑥2 ′′ where 𝑥2′ ≥ 0, 𝑥2′′ ≥ 0, replace 𝑥2 with 𝑥2 ′ − 𝑥2 ′′
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜
(𝑥1′ − 𝑥1′′ ) − 2(𝑥2 ′ − 𝑥2 ′′) − 3𝑥3
−(𝑥1′ − 𝑥1′′ ) + 3(𝑥2 ′ − 𝑥2 ′′) + 𝑥3 + 𝑥4
(𝑥1′ − 𝑥1′′ ) + 2(𝑥2 ′ − 𝑥2 ′′) + 3𝑥3
= 13
− 𝑥5 = 12
2(𝑥1′ − 𝑥1′′ ) − (𝑥2 ′ − 𝑥2 ′′) + 𝑥3
=4
𝑥1 ′ ≥ 0, 𝑥1 ′′ ≥ 0, 𝑥2 ′ ≥ 0, 𝑥2 ′′ ≥ 0,𝑥4 ≥ 0, 𝑥5 ≥ 0
Convert to >=0 𝑥3 ≤ −3
38
Linear Programming Format
• Converting to standard format:
– Variable 𝑥3 ≤ −3
• Let 𝑥3′ = −3 − 𝑥3 note that 𝑥3′ ≥ 0, 𝑥1′′ ≥ 0,
• Then we have 𝑥3 = −3 − 𝑥3′ so replace 𝑥3 with −3 − 𝑥3′
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜
(𝑥1′ − 𝑥1′′ ) − 2(𝑥2 ′ − 𝑥2 ′′) − 3(−3 − 𝑥3′ )
−(𝑥1′ − 𝑥1′′ ) + 3(𝑥2 ′ − 𝑥2 ′′) + (−3 − 𝑥3′ ) + 𝑥4
= 13
(𝑥1′ − 𝑥1′′ ) + 2(𝑥2 ′ − 𝑥2 ′′) + 3(−3 − 𝑥3′ ) − 𝑥5 = 12
2(𝑥1′ − 𝑥1′′ ) − (𝑥2 ′ − 𝑥2 ′′) + (−3 − 𝑥3′ )
=4
𝑥1 ′ ≥ 0, 𝑥1 ′′ ≥ 0, 𝑥2′ ≥ 0, 𝑥2′′ ≥ 0,𝑥3′ ≥ 0, 𝑥4 ≥ 0, 𝑥5 ≥ 0
39
Linear Programming Format
• Converted to standard format:
𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜
𝑥1′ − 𝑥1′′ − 2𝑥2′ + 2𝑥2′′ + 3𝑥3′
−𝑥1′ + 𝑥1′′ + 3𝑥2 ′ − 3𝑥2 ′′ − 𝑥3′ + 𝑥4
𝑥1′ − 𝑥1′′ + 2𝑥2 ′ − 2𝑥2 ′′ − 𝑥3′
2𝑥1′ − 2𝑥1′′ − 𝑥2 ′ + 𝑥2 ′′ − 𝑥3′
= 16
− 𝑥5 = 21
=7
𝑥1 ′ ≥ 0, 𝑥1 ′′ ≥ 0, 𝑥2′ ≥ 0, 𝑥2′′ ≥ 0,𝑥3′ ≥ 0, 𝑥4 ≥ 0, 𝑥5 ≥ 0
40
Linear Programming Format
• Let’s put it in matrix notation
𝑥1′
𝑥1′′
𝑥2′
−1 1
3 −3
– Let 𝐱 = 𝑥1′′ , 𝐀 = 1 −1 2 −2
2 −2 −1 1
𝑥3′
𝑥4
𝑥5
– Let 𝒄 = 1 −1 −2 2 3 0 0
−1 1 0
16
−1 0 1 , and 𝐛 = 21
−1 0 0
7
41
Linear Programming Format
• Standard form can have maximization objective as well but we will
discuss Simplex mostly for minimization problems
• Another format that can be helpful is canonical form
• Examples in Supplement 1
42
Next time…
• Linear algebra review
–
–
–
–
–
Basic linear algebra
Affine and Linear combinations
Linear independence, spanning
Convex sets and convex functions
Polyhedral sets, extreme points, representation
43