Privacy-Preserving Linear Programming
UCSD – Center for Computational Mathematics Seminar
January 11, 2011
Olvi Mangasarian
UW Madison & UCSD La Jolla
Problem Statement
• Entities with related data wish to solve a linear program
based on all the data
• The entities are unwilling to reveal their data to each other
– If each entity holds a different set of variables for all constraints,
then the data is said to be vertically partitioned
– If each entity holds a different set of constraints with all variables,
then the data is said to be horizontally partitioned
• Our approach: privacy-preserving linear programming
(PPLP) using random matrix transformations
– Provides exact solution to the total linear program
– Does not reveal any private information
Horizontally
Partitioned
Matrix
Linear
Programming
Constraint
Matrix
Vertically
Partitioned
Matrix
Variables
1 2 ..………….…………. n
1
2
.
.
Constraints
.
.
.
.
.
.
m
A1
A¢1
A¢2
AA
A3
A¢3
Outline
• Vertically (horizontally) partitioned linear
program
• Secure transformation via a random matrix
• Privacy-preserving linear program solution
• Computational results
• Summary
Vertically Partitioned Data:
Each entity holds different variables for the
same constraints
A¢1 A¢2 A¢3
A¢1
A¢2
A¢3
LP with Vertically Partitioned Data
We consider the linear program:
min c0 x where X = f x j Ax ¸ bg
x2X
where A 2 Rm£n, c 2 Rn
£ c0 ¤
The matrix A is divided into p vertical blocks
Each block is (m + 1) £ nj , j = 1; 2; : : : ; p,
such that n1 + n2 + : : : + np = n,
is \owned" by a distinct entity unwilling to make it public
Objective: Solve this linear program without revealing any
privately held data.
Secure Linear Program Generation
Each of the p entities chooses its own privately held
random matrixB¢j 2 Rk£nj , j = 1; : : : ; p,
where k ¸ n. De¯ne: B = [B¢1 B¢2 : : : : : : B¢p ] 2 Rk£n .
We note immediately that the rank of the random matrix
B 2 Rk£n with k ¸ n is n, which is the reason
for choosing k ¸ n. Utilizing this fact we de¯ne
the invertible transformation: x = B 0 u, and
its least 2-norm inverse: u = B(B 0 B)¡1 x:
We now transform our original linear program into
the following \secure" linear program:
min c0 B 0 u where U = f u j AB 0 u ¸ bg:
u2U
Why Secure Linear Program?
We use the term \secure" to describe the transformed
linear program because it does not reveal any of
£ c0 ¤
the privately held data Aj¢j , j = 1; : : : ; p. This is so
because for each entity di®erent from entity j, it is
impossible to compute either cj from the revealed product
cj 0 B¢j 0 , or A¢j from the revealed product A¢j B¢j 0
without knowing the random matrix B¢j chosen by
entity j and known to itself only.
Original & Secure LPs Are Equivalent
Let k ¸ n for the random matrix B 2 Rk£n .
The secure linear programis solvable if and only if
the original linear program is solvable in which case
the extrema of both linear programs are equal.
For one page proof see:
ftp:/ / ftp.cs.wisc.edu/ pub/ dmi/ tech-reports/ 10-01.pdf
Optimization Letters, to appear.
PPLP Algorithm
1. All p entities agree on a k ¸ n, the number of rows
of the random matrix B 2 Rk£n as de¯ned earlier.
2. Each entity generates its own privately held random
matrix B¢j 2 Rk£nj , j = 1; : : : : : : ; p, where nj
is the number of features held by entity j. This results
in B = [B¢1 B¢2 : : : : : : B¢p ] 2 Rk£n :
PPLP Algorithm (Continued)
3. Each entity j makes public only its matrix product
A¢j B¢j 0 as well as its cost coe±cient product B¢j cj .
These products do not reveal either A¢j or cj but
allow the public computation of the full constraint
matrix needed for the secure linear program:
AB 0 = A¢1 B¢1 0 + A¢2 B¢2 0 + : : : : : : + A¢p B¢p 0 ;
as well as the cost coe±cient:
c0 B 0 = c01 B¢1 0 + c02 B¢2 0 + : : : : : : + c0p B¢p 0 :
PPLP Algorithm (Continued)
4. A public optimal solution vector u to the secure
linear program and a public optimal objective function
value c0 B 0 u are computed. This optimal value equals
the optimal objective function of the original linear
program.
5. Each entity computes its optimal xj component group
as follows:
xj = B¢j 0 u; j = 1; : : : ; p:
PPLP Algorithm (Continued)
6. The solution component vectors xj , j = 1; : : : ; p,
are revealed by its owners if a public solution vector to
the original linear program is agreed upon. Else,
the component vectors may be kept private if only
the minimum value is needed, in which case that
minimum value equals the publicly available minimum
value min c0 B 0 u of the secure linear program.
u2U
Computatinal Results
Example 1 (k=n=1000)
² Generated a random solvable linear program
with m = 100 and n = 1000
² Partitioned the columns of A as well as the cost vector c
into three groups with n1 = 500, n2 = 300 and n3 = 200
² Generated three random matrices, with coe±cients
uniformly distributed in the interval [0,1] with
B¢1 2 Rn£n1 , B¢2 2 Rn£n2 andB¢3 2 Rn£n3
² Solved secure linear program and compared its optimal
objective value with that of the original linear program.
² The two optimal objectives agreed to 14 signi¯cant
¯gures attained at two distinct optimal solution points.
Computatinal Results
Example 2 (k=n=100)
² Generated a random solvable linear program
with m = 1000 and n = 100
² Partitioned the columns of A as well as the cost vector c
into three groups with n1 = 50, n2 = 30 and n3 = 20
² Generated three random matrices, with coe±cients
uniformly distributed in the interval [0,1] with
B¢1 2 Rn£n1 , B¢2 2 Rn£n2 andB¢3 2 Rn£n3
² Solved secure linear program and compared its optimal
objective value with that of the original linear program.
² The two optimal objectives agreed to 13 signi¯cant
¯gures attained at points that were essentially the same,
the 1 -norm of their di®erence was less than 2:3e ¡ 13.
Horizontally Partitioned Constraint Matrix:
Entities hold different constraints with the same variables
A1
A2
A3
A1
A2
A3
LP with Horizontally Partitioned Data
We consider the linear program:
min c0 x where X = f x j Ax = b; x ¸ 0g
x2X
where A 2 Rm£n, c 2 Rn
Here, [A b], is divided into p horizontal blocks of
m1 ; m2 ; : : : : : : and mp , (n + 1)-dimensional
rows with m1 + m2 + : : : + mp = m.
Each block of rows of [A b] is \owned" by a distinct
entity that is unwilling to make its block of data public.
Objective: Solve this linear program without revealing any
privately held data.
Secure Linear Program Generation
Each of the p entities chooses its own privately held
random matrix B¢i 2 Rk£mi , i = 1; : : : ; p, where k ¸ m,
which de¯nes: B = [B¢1 B¢2 : : : : : : B¢p ] 2 Rk£m :
We note immediately that the rank of the random matrix
B 2 Rk£m with k ¸ m is m, which is the reason
for choosing k ¸ m. Utilizing this fact we de¯ne
the transformations: BA and Bb.
We now transform our original linear program into
the following \secure" linear program:
min c0 y where Y = f y j BAy = Bb; y ¸ 0g:
y2Y
Why Secure Linear Program?
We use the term \secure" to describe the transformed
linear program because it does not reveal any of the
privatelyheld data: the horizontal partitions of [A b].
This is so because for each entity di®erent from entity i,
it is impossible to compute either Ai from the revealed
product B¢i Ai , or bi from the revealed product B¢i bi
without knowing the random matrix B¢i
chosen by entity i and known to itself only.
Original & Secure LPs Are Equivalent
Let k ¸ m for the random matrix B 2 Rk£m .
The secure linear programis solvable if and only if
the original linear program is solvable in which case
the extrema of both linear programs are equal.
The proof follows from the obvious fact that:
Ax = b ( )
BAx = Bb:
For more details see:
ftp:/ / ftp.cs.wisc.edu/ pub/ dmi/ tech-reports/ 10-02.pdf
PPHPLP Algorithm
1. All p entities agree on a value for k ¸ m, where k
is the number of rows of the random matrix B 2 Rk£m
2. Each entity generates its own privately held random
matrix B¢i 2 Rk£mi , i = 1; : : : : : : ; p,where mi is
the number of rows held by entity i which results in:
B = [B¢1 B¢2 : : : : : : B¢p ] 2 Rk£m :
PPHPLP Algorithm (Continued)
3. Each entity i makes public only its matrix product
B¢i Ai as well as its right hand side product B¢i bi .
These products do not reveal either Ai or bi but allow
the public computation of the full constraint matrix
needed for the secure linear program:
BA = [B¢1 A1 + B¢2 A2 + : : : + B¢p Ap ] 2 Rk£n ;
as well as the right hand side :
Bb = [B¢1 b1 + B¢2 b2 + : : : + B¢p bp ] 2 Rk :
4. A public optimal solution vector y to the secure linear
program is obtained which also solves the original linear
program
Computatinal Results
Example 1 (k=1000)
² Generated a random solvable linear program
with m = 600 and n = 1000
² Partitioned the rows of A as well as the right hand
side vector b into three groups with m1 = 100, m2 = 200
and m3 = 300.
² Generated three random matrices, with coe±cients
uniformly distributed in the interval [0,1] with
B1 2 Rk£m1 , B2 2 Rk£m2 and B3 2 Rk£m3 .
² Solved secure linear program and compared its optimal
solution with that of the original linear program.
² The two optimal solutions were identical.
Computatinal Results
Example 2 (k=1000)
² Generated a random solvable linear program
with m = 1000 and n = 1000
² Partitioned the rows of A as well as the right hand
side vector b into three groups with m1 = 200, m2 = 300
and m3 = 500.
² Generated three random matrices, with coe±cients
uniformly distributed in the interval [0,1] with
B1 2 Rk£m1 , B2 2 Rk£m2 and B3 2 Rk£m3 .
² Solved secure linear program and compared its optimal
solution with that of the original linear program.
² The two optimal solutions were identical.
Summary & Outlook
Privacy preserving linear programming
for vertically or horizontally partitioned data
– Based on a transformation using a random
matrix B
– Get exact solution to the original linear
program without revealing privately held data
Possible extensions to: horizontally partitioned inequality
constraints, complementarity problems and nonlinear
programs
References
ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/10-01.pdf
ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/10-02.pdf
Optimization Letters, to appear