1
Learning to Succeed while Teaching to Fail:
Privacy in Closed Machine Learning Systems
arXiv:1705.08197v1 [stat.ML] 23 May 2017
Jure Sokolić, Qiang Qiu, Miguel R. D. Rodrigues, and Guillermo Sapiro
Abstract
Security, privacy, and fairness have become critical in the era of data science and machine learning.
More and more we see that achieving universally secure, private, and fair systems is practically impossible.
We have seen for example how generative adversarial networks can be used to learn about the expected
private training data; how the exploitation of additional data can reveal private information in the original
one; and how what looks like unrelated features can teach us about each other. Confronted with this
challenge, in this paper we open a new line of research, where the security, privacy, and fairness is
learned and used in a closed environment. The goal is to ensure that a given entity (e.g., the company or
the government), trusted to infer certain information with our data, is blocked from inferring protected
information from it. For example, a hospital might be allowed to produce diagnosis on the patient (the
positive task), without being able to infer the gender of the subject (negative task). Similarly, a company
can guarantee that internally it is not using the provided data for any undesired task, an important goal
that is not contradicting the virtually impossible challenge of blocking everybody from the undesired
task. We design a system that learns to succeed on the positive task while simultaneously fail at the
negative one, and illustrate this with challenging cases where the positive task is actually harder than
the negative one being blocked. Fairness, to the information in the negative task, is often automatically
obtained as a result of this proposed approach. The particular framework and examples open the door to
security, privacy, and fairness in very important closed scenarios, ranging from private data accumulation
companies like social networks to law-enforcement and hospitals.
J. Sokolić and M. R. D. Rodrigues are with the Department of Electronic and Electrical Engineering, Univeristy College
London, London, UK (e-mail: {jure.sokolic.13, m.rodrigues}@ucl.ac.uk).
Q. Qiu and G. Sapiro are with the Department of Electrical and Computer Engineering, Duke University, NC, USA (e-mail:
{qiang.qiu, guillermo.sapiro}@duke.edu).
The work of Guillermo Sapiro was partially supported by NSF, ONR, ARO, NGA.
May 24, 2017
DRAFT
2
I. I NTRODUCTION
Advances in machine learning and data science allow to develop product and services that leverage
individuals data to provide valuable services. At the same time care must be taken to provide an appropriate
amount of protection for users in order to adhere to various legal and ethical constraints.
In this work we envision the following scenario: An entity (e.g., the company or the government) wants
to offer a service to its users based on their data. However, some users want to be able to prevent the
entity from inferring certain sensitive information from their data. The need for this may come from
security, fairness, or privacy concerns. For example, a hospital might be allowed to produce diagnosis
on the patient, without being able to infer some other irrelevant condition of the subject. We may allow
identity verification for security purposes, but we want to be sure that we can not be discriminated based
on gender, race, or age, guaranteeing that such information can not even be inferred from the provided
data. This motivates the main question that we try to answer in this work:
How can we design a system that can provide a valuable service to a user and offer protection to their
sensitive data at the same time?
The benefits of such a system are twofold. First, the users will be more comfortable because their
privacy is respected for the tasks the user wants to block. Second, the system will offer the entity offering
the service a principled approach to prove that they are not using/inferring users’ sensitive information in
case of legal/ethical disputes.
Before describing the proposed framework we need to further address two issues that motivate our
proposed system. First, can we design a system that is universal, meaning capable to prevent the undesired
task by any other system? And second, is the problem that we are trying to solve trivial? The negative
answers to these questions are important motivations to the framework here introduced.
Regarding the first question, there are many examples in practice where additional sources of information
lead to attacks on the systems that were designed to be private. For example, the authors in [1] showed
how to break the anonymity of the publicly released Netflix Prize Dataset. They achieved that by using
other public datasets and an appropriate algorithm. It has been shown in [2] that a person in an image
can be identified based on the social network graph and the photos of other users even when the faces in
the images are obfuscated. The source and power of this additional data may be very hard to anticipate.
Another example is the unexpected possibility to infer cardio-vascular health, a task we might want to
protect, from regular credit scores as those commonly provided to get mortgage authorization [3].
The impossibility of universal privacy protection just exemplified has also been studied extensively in
the domain of differential privacy [4], where a number of authors have shown that assumptions about
May 24, 2017
DRAFT
3
the data or the adversary must be made in order to be able to provide utility [5]–[7]. Similarly, in the
fairness domain the authors in [8] propose a method to remove discrimination from any learned predictor
and note that their fairness objective is not universal, but rather domain specific. As a consequence their
framework does not aim to certify fairness but rather provide a set of tools which can be used to improve
the fairness of a system.
On the other hand, one may argue that users can prevent inference of sensitive variables by simply
sharing less data. For example, [9] shows that if the feature vector representing a face image is sufficiently
low dimensional, one can infer the gender of the person, but not his/her identity. However, one may
also be interested in face verification, which is not possible if the features do not provide sufficient
information, and be agnostic to gender at the same time. This task is much harder, meaning the desired
task (verification) is harder than the protected task (gender detection), and the appropriate assumptions
must be made, as we will be shown in the sequel.
In view of the issues presented above we propose a novel framework that rather than being universal,
assumes a closed environment where privacy with respect to particular sensitive variables is desired. The
goal is to ensure that a given entity, trusted to infer certain information with our data, is blocked from
inferring sensitive information from it. For example, a hospital might be allowed to produce diagnosis on
the patient (the positive task), without being able to infer the irrelevant gender of the subject (negative
task). Similarly, a company can guarantee they internally are not using the provided data for any undesired
task, an important goal that is not contradicting the virtually impossible challenge of blocking everybody
from the undesired task.
To achieve this, we assume an environment where the entity declares a set of (training) data and a set
of tools (machine learning algorithms) that will be used to provide a certain service (positive task) to
users. We then design a data sanitization function whereby users provide data to the entity that prevents
inference of sensitive variables (negative tasks) within the constrained environment. This general idea is
visualized in Figure 1.
The proposed framework is describe in detail in Section II. A critical application is presented in
Section III. Related literature is reviewed and discussed in Section IV. Finally, the paper is concluded in
Section V. Details on the experimental procedures are provided in the Appendix.
II. P ROPOSED F RAMEWORK
Our framework consists of a closed environment with a set of data Z and a set of algorithms A. In
particular, we assume that the training set consist of m training examples: Z = {zi }m
i=1 . An element of
Z is a triplet z = (x, t, s), where x ∈ X represents a feature vector is some feature space X , t is a label
May 24, 2017
DRAFT
4
Fig. 1. The entity provides a service to its users in a closed environment with a set of data and a set of algorithms. Given a
sensitive variable that may be infered from raw user data, the entity designs a data sanitization function, which transformes
users’ data in such a way that the sensitive variables may not be infered whithin the closed environment. The data sanitization
function is then either shared with the user who can now share sanitized data or it is applied to the data before the users’ data
enters the entity.
associated with utility (positive task) and s is a label associated with sensitive data (negative task). For
the sake of simplicity we assume that t and s are binary, i.e., t, s ∈ {−1, 1}. The set of algorithms A
consists of algorithms A that map the training set to a predictor
A : Z → qA(Z)
A ∈ A,
(1)
where the predictor is of the form
qA(Z) : X → {−1, 1} .
(2)
The training data and the algorithms are used to provide a service to the users, where the users’ data
consists of feature vectors x and it is denoted by U = {(x)i }ni=1 . In particular, the entity provides a
service to the user via the prediction of the utility variable t. However, since the feature vector may
also be used to predict the sensitive variable s, care must be taken to provide protection against that. As
discussed before, on one hand, a user might not use the service because of the concerns of revealing her
sensitive information. On the other hand, the entity wants to be able to prove that they are not misusing
users data by predicting the sensitive variable s against the user’s preference, or for example due to
fairness. We assume that the training set has no privacy restrictions.
As discussed in the Introduction, giving guarantees about non-identifiability of certain variables in the
“open world” where new data or new algorithms are readily available is very likely impossible. Therefore,
what we want to do is to design mechanisms that will prevent prediction of sensitive variables within this
important closed environment. Such methods are very relevant in practice: i) they create an additional
May 24, 2017
DRAFT
5
level of trust between the users and the entity; ii) it gives the entity an elegant way to certify that sensitive
information is not being abused within their closed environment; and iii) can provide additional value
such as fairness.1
We require the entity to define a utility metric, a privacy metric, and to design a data sanitization
function that will achieve privacy while providing utility. Of particular challenge is the case when inferring
s (private data) is actually easier than inferring the actual authorized information t. This is here addressed
for the first time.
a) Utility Metric, Privacy Metric and Data Sanitization.: First, we will assume that the entity has
a test set Z 0 with m0 samples where the samples are drawn from the same distribution as the training
examples. We define both the utility metric and the privacy metric based on the testing set. This is needed
since a representative test set is exploited to gain confidence about the utility of the machine learning
model. The same test can be used to verify that given the training set Z and the set of algorithms A, no
meaningful conclusion about the sensitive variable s can be drawn from the data.
The entity will then design a data sanitization function
f :X →X,
(3)
such that both utility and privacy will be satisfied. Note that f maps to the same feature space X so that
sanitized data f (x) may be used as the input to a predictor qA(Z) . We will evaluate the data sanitization
function on the sanitized test set which will be denoted as Zf0 :
Zf0 = {(f (x), s, t)) : (x, s, t) ∈ Z 0 }.
(4)
In particular, the utility metric of the closed system will be denoted by
u(A, Z, Zf0 ) ∈ [0, 1] ,
(5)
and the privacy metric will be denoted as
p(A, Z, Zf0 ) ∈ [0, 1] .
(6)
The goal is to design such f that both utility and privacy metrics are as close to 1 as possible.
For example, denote by CAt (A, Z, Zf0 ) the classification accuracy of the utility task of the predictor
qA(Z) trained on the training set Z and evaluated on the sanitized test set Zf0 :
X
1
CAt (A, Z, Zf0 ) = 0
1 − `(qA(Z) (f (x)), t) ,
|Zf |
0
(7)
(f (x),s,t)∈Zf
1
Such a closed environment can be achieved for example by specifying what algorithm has access to what data in a manner
similar to the implementation of human access control in digital environments.
May 24, 2017
DRAFT
6
where ` is a 0-1 loss. Then we might define the utility metric as the classification accuracy of the best
possible predictor given the training set Z and the set of algorithms A:
u(A, Z, Zf0 ) = max CA(A, Z, Zf0 ) .
A∈A
(8)
Similarly, denote by CAs (A, Z, Zf0 ) the classification accuracy of the sensitive task of the predictor qA(Z)
trained on the training set Z and evaluated on the sanitized test set Zf0 :
CAs (A, Z, Zf0 ) =
1
|Zf0 |
X
1 − `(qA(Z) (f (x)), s) .
(9)
(f (x),s,t)∈Zf0
Assume also that the prior probabilities of the sensitive labels are balanced, p(s = 1) = p(s = 0) = 0.5.
Then the privacy metric can be defined as the deviation of the the most accurate predictor given the
training set Z and the set of algorithms A from the “random” classifier:
p(A, Z, Zf0 ) = min 1 − 2 · |CAs (A, Z, Zf0 ) − 0.5| .
A∈A
(10)
Note that the definitions of privacy and the utility are application specific. For example, we may define
the utility or privacy metrics via averages over different users. However, we could use more restrictive
definitions where utility or privacy are defined per user. The novel framework here introduced is independent
of the particular selection. A concrete example is presented in Section III.
The entity might now collect users data via the data sanitization function f into the environment
with training data Z and with the set of algorithms A. The closed environment is certified with the
utility u(A, Z, Zf0 ) and the privacy p(A, Z, Zf0 ), which may be communicated to the user or a possible
regulatory entity.
We have presented a framework to protect users privacy based explicitly on the predefined data and
algorithms. The guarantees do not extend to algorithms outside A and additional data (side information)
outside Z . Although these properties would be desirable, as previously described a large body of literature
suggests that universal privacy protection is only possible at the expense of utility.
III. S AMPLE A PPLICATIONS
We now demonstrate the proposed framework with an application involving face images and different
types of closed environments.
May 24, 2017
DRAFT
7
A. Face verification and gender recognition from face images
We will focus on the tasks of face verification2 and gender classification. In particular, we will
consider the face verification as the utility task and the gender recognition as the sensitive task. This
may be useful in various applications where we want to re-identify users, but want to prevent gender
discrimination/information within the system. This of course goes beyond fairness, where the goal is
only to make sure that the system’s performance is independent of gender, but not to block the gender
detection.
Note that for the opposite case, where gender classification is the utility task and face verification is the
sensitive task is also relevant. However, in this case the undesired task can be easily blocked by simply
using a sufficiently low dimensional projection, as shown in [9], and is therefore less challenging.
1) Dataset and Tools:
a) Dataset.: We use the FaceScrub dataset [10], containing approximately 100,000 face images of
530 individuals, with 265 males and 265 females. For each image x is obtained by using the features
of the last layer of the VGG-face network [11] projected to dimension 1000 using principal component
analysis. Associated with each feature is the identity label t and the gender label s. Note that our positive
task is verification and not identification, therefore, given two examples (x, t, s) and (x0 , t0 , s0 ) we want to
establish either t = t0 or t 6= t0 based on the features x and x0 . The negative task is gender classification
where the features x are used to predict the gender s.
We create three subsets from the original dataset:
•
Training set Z = {(x, t, s)i }m
i=1 , which represents data in the closed environment and is used to
design the data sanitization function;
•
0
Test set Z 0 = {(x, t, s)i }m
i=1 , which is used to evaluate the utility and privacy metrics. The sanitized
0
version of the test set is denoted as Zf0 = {(f (x), t, s)i }m
i=1 ;
•
User set U = {(x)i }ni=1 , which represents users’ data.
The three sets Z , Z 0 and U have a balanced ratio of males and females and do not share identities.
b) Metrics.: We use the Area Under the Curve (AUC) of the Receiver Operating Characteristic
(ROC) curve to measure success of the face verification task and the gender classification accuracy (CA)
to measure success of the gender classification task. Therefore, the utility and the privacy metrics for this
particular application are defined as
u(A, Z, Zf0 ) = AUC(Zf0 )
2
(11)
Note that face verification verifies if two images belong belong to the same person (consider for example the ID verification
at the airport or the entrance to a facility). It does not identify a person and therefore does not reveal gender.
May 24, 2017
DRAFT
8
and
p(A, Z, Zf0 ) = min 1 − 2 · |CAs (A, Z, Zf0 ) − 0.5| ,
A∈A
(12)
respectively.
c) Algorithms.: We test using 3 algorithms, helping us to present the proposed framework and not
just an instance of it:
•
Verification algorithm (VA): it takes two feature vectors and it outputs the cosine similarity between
the two vectors;
•
Support Vector Machine (SVM) classifier;
•
Random Forest (RF) classifier.
The exact training procedure and hyper-parameter settings are described in the Appendix.
d) Data Sanitization.: We consider two data sanitization techniques that are fully described in the
Appendix:
•
Data Sanitization with Linear Projection:
f (x) = P x ,
(13)
where P is a linear projection matrix.
•
Data Sanitization with Maximum Mean Discrepancy Transformation:
f (x) = x + n(x) ,
(14)
where n(x) is a “noise” vector that depends non-linearly on x.
The data sanitization function are designed so that they maximize the privacy and minimally affect the
utility.
2) Results: We now explore various scenarios where we prevent gender classification without compromising verification within a closed environment. In all the cases we will assume that the training set is
part of the closed environment. We will then consider the different examples of data sanitization functions
and the different algorithms allowed in the closed environment.
a) Baseline.: First, we consider a baseline scenario, where the closed environment uses the training
set Z and all three algorithms: A = {VA, SVM, RF} and does not do any data sanitization. We report
the values of the utility metric and the privacy metric as measured on the test set Z 0 . We also report the
gender classification accuracy of the SVM classifier, the gender classification accuracy of the RF classifier
and the verification AUC on the user data. The results are reported in Table I.
May 24, 2017
DRAFT
9
TABLE I
BASELINE . S ETUP : f (x) = x, Z , A = {VA, SVM, RF}.
User data
Utility metric
Privacy metric
Gender acc. (SVM)
Gender acc. (RF)
Verification AUC
0.9607
0.0672
96.73 %
95.37 %
96.10 %
TABLE II
C ASE 1. S ETUP : f (x) = P x, Z , A = {VA, SVM}.
User data
Utility metric
Privacy metric
Gender acc. (SVM)
Gender acc. (RF)
Verification AUC
0.9600
0.8892
45.81 %
/
96.08 %
TABLE III
C ASE 2. S ETUP : f (x) = x + n(x), Z , A = {VA, SVM, RF}.
User data
Utility metric
Privacy metric
Gender acc. (SVM)
Gender acc. (RF)
Verification AUC
0.9493
0.8713
50.27 %
53.84 %
95.07 %
Note that the utility metric is close to 1 and the privacy metric is close to zero. This is expected as no
data sanitization is used. The gender classification accuracies on the data in the user set are high, which
implies low privacy, and the verification AUC on the user data is also high, which implies high utility.
b) Case 1: Linear sanitization and SVMs.: Here we set the closed environment to include the
training set Z , algorithms: A = {VA, SVM}, and the linear sanitization function for the users’ data. The
values of the utility metric and the privacy metric as measured on the test set Zf0 are reported in Table II.
The gender classification accuracy of the SVM classifier and the verification AUC on the user data are
reported as well. Note that the the gender classification accuracy of the RF classifier is not reported
because RF classifier is not in the closed environment.
The utility metric in this case if very close to the baseline. Therefore, the utility is preserved by the data
sanitization function. The privacy metric, on the other hand, is much higher than in the baseline example,
which means that gender recognition does not perform well on the users’ data. These observations are also
supported by a low gender classification accuracy of the SVM classifier and the verification AUC on the
May 24, 2017
DRAFT
10
user dataset, which dropped for only 0.02% compared to the baseline. Therefore, we have simultaneously
achieved privacy (and gender fairness as a consequence) and utility within the specified environment.
c) Case 2: MMD sanitization, SVMs and RFs.: Now we extend the closed environment from case
1 to include the stronger RF classifiers: A = {VA, SVM, RF}. The linear sanitization function is not
effective when used with the RF classifiers, therefore we use the MMD data sanitization. The results
following the format of the baseline and case 1 are presented in Table III.
Note that the utility metric and the privacy metric are slightly lower than in the case 1. Nevertheless,
the verification AUC on the user dataset dropped by 1.03% compared to the baseline, whereas the SVM
and RF gender classifiers trained in the closed environment achieved accuracies 50.27% and 53.84%,
respectively. We can claim that the data sanitization is therefore effective.
IV. R ELATED W ORK
While the concept of closed environment privacy as here defined is new, and the formulation presented
in the previous section should be considered as one possible realization of it, it is related to other
contributions in the literature. We give a review of some of these works next.
a) Privacy.: The privacy problem has been widely studied in the data mining and database
communities, where differential privacy [4] is one of the most popular approaches.
Differentially private data release mechanisms ensure that an adversary may not establish a presence or
absence of an individual in a database [12]. An important property, emphasized in the differential privacy
literature, is the fact that there is no need for assumptions about the side information of an adversary.
However, such universality comes at an expense of utility, as shown in [5]–[7]. Our proposed framework
differs from the differential privacy framework by assuming a closed environment where the data and
tools are predefined/pre-trained. A consequence is that we can maintain utility, while still offering privacy
within the closed environment.
Differential privacy has also been applied to training of machine learning models where the goal
is to keep the training data private while learning the parameters of the model, e.g., [13], [14]. A
successful attack using generative adversarial networks on differentialy private training mechanisms has
been proposed in [15]. Our framework differs from these models in the sense that we propose data
sanitization mechanisms and privacy for user’s data and not for training data.
The Pufferfish framework [7] defines privacy with respect to a set of potential secrets and with respect
to a set of assumptions about the knowledge of a potential adversary. Similarly to our framework, this
framework constrains the problem to a particular secret and assumes an adversary has a limited prior
knowledge. In our framework we explicitly constraint a potential adversary by defining the training data
May 24, 2017
DRAFT
11
and the set of algorithms. Moreover, we also propose two techniques for data sanitization that are based
on data and achieve privacy and utility within a closed environment.
A deep learning architecture for privacy-preserving mobile analytics has been proposed in [9]. Their
main tools is dimensionality reduction of shared features, which allows them to prevent the success
of hard tasks (person identification), where a higher feature dimensionality is required. However, such
methods do not work if we want to disable an easier task (e.g, gender classification).
b) Fairness.: The goal of fairness in machine learning is to ensure that predictions are not biased or
discriminatory [16]. To achieve fairness we may regularize the feature representations [17]–[19] or design
predictors in a way that they obey the fairness constraints [8], [20].
A fair representation with respect to a certain variable does not necessarily imply privacy or nonidentifiability of this variable. On the other hand, fairness can automatically result from the here proposed
framework. Certain approaches such as the variational fair auto-encoder [19] or a censored representation
trained using an adversary [18] also achieve non-identifiability. These methods can potentially be used in
our proposed framework for closed privacy. However, our focus is not on a particular method or algorithm
but rather on a wider framework that may encompass various algorithms.
c) Other.: An orthogonal direction that offers privacy is homomorphic encryption [21], where models
are trained and tested using encrypted data, and the prediction results are also returned in an encrypted
form. Homomorphic encryption has been applied to neural networks in [22]. Such homomorphic encryption
schemes are extremely computationally inefficient. Our proposed framework is as efficient as the desired
utility, and again its goal is to block the entity from extracting protected information and not to block
others from accessing data (encryption).
V. C ONCLUSIONS
In this work we have introduced a framework that provides privacy to users within a closed environment.
We have also demonstrated its application on face image data. There are many possible extension to the
presented framework. In particular, it may be interesting to design methods that would allow sharing of
users’ data between various closed environments in a way consistent with privacy guarantees. Another
relevant extension is to adaptive environments where users’ data is used internally to improve the models.
Finally, in this work we measure utility and privacy empirically, based on a test set, which is perfectly
reasonable from the practical perspective. However, we also see opportunities for a more detailed theoretical
study that would lead to guarantees in a stronger statistical sense.
May 24, 2017
DRAFT
12
A PPENDIX
a) Details of SVM and RF training.: The training procedure of the classifier works as follows:
Provided data is randomly split into training and validation sets. The predictor is trained on the training
set for various hyper-parameter settings and the performance on the validation set is recorded. The
hyper-parameters corresponding to the best performance on the validation set are chosen and the predictor
is trained on the entire dataset.
We use the linear SVM and RF implemented in [23]. The constant C associated with the linear SVM
is picked from the set {1e − 3, 1e − 1, 1, 100}. For the RF classifiers we consider number of trees in
{25, 50, 100} and tree depth is set to 3, 5 or it is set automatically by the algorithm.
b) Data Sanitization with Linear Projection.: The sanitization function with linear projection P is
defined as
f (x) = P x .
(15)
The linear projection P is obtained by iterating the following steps, where P is set to identity initially:
1) The features x in the training set Z and the test set Z 0 are transformed by P : x = P x.
2) An SVM classifier is trained on the training set Z to predict the sensitive variable s. SVM training
details are provided in the paragraph above.
3) A linear projection P 0 is constructed so that the kernel of the projection corresponds to the weight
vector of the trained SVM. In this way the projection “collapses” the dimension of the space where
the training samples are linearly separable.
4) Projection P is updated to include the kernel of the projection P 0 : P = P 0 P .
The process is repeated until we achieve a sufficiently low classification accuracy on the test set (we used
55% in our case).
The trade-off between utility and privacy is achieved by increasing or decreasing the dimension of the
kernel of the projection P .
c) Data Sanitization with Maximum Mean Discrepancy Transformation.: We extend the technique
proposed in [24], which uses the Maximum Mean Discrepancy (MMD) statistics [25]. It is originally
used for image style transfer using convolutional neural networks.
The sanitization function is defined as
f (x) = x + n(x) ,
(16)
where n(x) is obtained as follows: First a dictionary D is constructed from the feature vectors x in the
training set Z . We set n(x) to be a linear combination of the features in the training set: n(x) = Dθ(x)
May 24, 2017
DRAFT
13
where θ(x) is the vector of coefficients. The central part of the method is the computation of the parameter
vector θ(x), which we describe next.
We use the features x from the training set Z to construct two subsets: X+ = {x : (x, s, t) ∈ Z, s = 1}
and X− = {x : (x, s, t) ∈ Z, s = −1}, which correspond to the two classes associated with the sensitive
label. Then for each user’s feature vector x we determine randomly the target class s0 ∈ {−1, 1} and
define the cost function


c(θ(x)) = s0 × 
1
|X+ |
X
k(x+ , x + Dθ(x)) −
x+ ∈X+
1
|X− |
X
k(x− , x + Dθ(x)) ,
(17)
x− ∈X −
where k(x, x0 ) is the kernel function. We choose k(x, x0 ) to be a Gaussian kernel, k(x, x0 ) = exp(−kx −
x0 k22 /2σ 2 ) with parameter σ = 0.001 in our experiments.
The value of θ(x) is obtained by maximizing the cost function c(θ(x)):
max c(θ(x)) .
(18)
θ(x)
The optimization problem is solved by gradient descent using step size 0.05 and 10,000 iterations.
Intuitively, a large positive value of
X
X
1
1
k(x+ , x + Dθ(x)) −
k(x− , x + Dθ(x))
|X+ |
|X− |
x+ ∈X+
(19)
x− ∈X −
indicates that x + D(θ) resembles the data in X+ , and a large negative value of (19) indicates that
x + Dθ(x) resembles the data in X− . Therefore, the optimization problem in (18), where s0 ∈ {−1, 1},
can be interpreted as a non-linear noise model that changes feature x to be close to either |X+ | or |X− |.
R EFERENCES
[1] A. Narayanan and V. Shmatikov, “Robust de-anonymization of large sparse datasets,” IEEE Symposium on Security and
Privacy, pp. 111–125, 2008.
[2] S. J. Oh, R. Benenson, M. Fritz, and B. Schiele, “Faceless person recognition; Privacy implications in social media,”
European Conference on Computer Vision (ECCV), pp. 19–35, 2016.
[3] S. Israel, A. Caspi, D. W. Belsky, H. Harrington, S. Hogan, R. Houts, S. Ramrakha, S. Sanders, R. Poulton, and T. E.
Moffitt, “Credit scores, cardiovascular disease risk, and human capital,” Proceedings of the National Academy of Sciences,
vol. 111, no. 48, pp. 17 087–17 092, 2014.
[4] C. Dwork, “Differential privacy: A survey of results,” International Conference on Theory and Applications of Models of
Computation, pp. 1–19, 2008.
[5] C. Dwork and M. Naor, “On the difficulties of disclosure prevention in statistical databases or the case for differential
privacy,” Journal of Privacy and Confidentiality, vol. 2, no. 1, pp. 93–107, 2010.
[6] D. Kifer and A. Machanavajjhala, “No free lunch in data privacy,” ACM SIGMOD International Conference on Management
of data, pp. 193–204, 2011.
May 24, 2017
DRAFT
14
[7] ——, “Pufferfish: A framework for mathematical privacy definitions,” ACM Transactions on Database Systems, vol. 39,
no. 1, pp. 3:1–3:36, 2014.
[8] M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” Advances in Neural Information
Processing Systems (NIPS), pp. 3315–3323, Oct. 2016.
[9] S. Ali Ossia, A. Shahin Shamsabadi, A. Taheri, H. R. Rabiee, N. D. Lane, and H. Haddadi, “A hybrid deep learning
architecture for privacy-preserving mobile analytics,” arXiv:1703.02952.
[10] H. Ng and S. Winkler, “A data-driven approach to cleaning large face datasets,” IEEE International Conference on Image
Processing (ICIP), pp. 343–347, 2014.
[11] O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” British Machine Vision Conference, pp. 1–12, 2015.
[12] L. Wasserman and S. Zhou, “A statistical framework for differential privacy,” Journal of the American Statistical Association,
vol. 105, no. 489, pp. 375–389, 2010.
[13] M. J. Wainwright, J. I. Michael, and J. C. Duchi, “Privacy aware learning,” Advances in Neural Information Processing
Systems (NIPS), pp. 1430–1438, 2012.
[14] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential
privacy,” ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318, Jul. 2016.
[15] B. Hitaj, G. Ateniese, and F. Perez-Cruz, “Deep models under the GAN: Information leakage from collaborative deep
learning,” arXiv:1702.07464, 2017.
[16] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” ACM Innovations in Theoretical
Computer Science Conference, pp. 214–226, 2012.
[17] R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” International Conference on
Machine Learning (ICML), pp. 325–333, 2013.
[18] H. Edwards and A. Storkey, “Censoring representations with an adversary,” International Conference on Learning
Representations (ICLR), pp. 1–14, Nov. 2016.
[19] C. Louizos, K. Swersky, Y. Li, M. Welling, and R. Zemel, “The variational fair autoencoder,” International Conference on
Learning Representations (ICLR), pp. 1–11, Nov. 2016.
[20] M. B. Zafar, M. Gomez Rodriguez, and K. P. Gummadi, “Fairness constraints : A mechanism for fair classification,”
arXiv:1507.05259, 2017.
[21] T. Graepel, K. Lauter, and M. Naehrig, “ML confidential: Machine learning on encrypted data,” International Conference
on Information Security and Cryptology, pp. 1–21, 2012.
[22] N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, and J. Wensing, “Cryptonets: Applying neural networks to
encrypted data with high throughput and accuracy,” International Conference on Machine Learning (ICML), pp. 201–210,
2016.
[23] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg,
J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in
Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
[24] J. R. Gardner, P. Upchurch, M. J. Kusner, Y. Li, K. Q. Weinberger, K. Bala, and J. E. Hopcroft, “Deep manifold traversal:
Changing labels with convolutional features,” arXiv:1511.06421, 2015.
[25] A. Gretton, K. M. Borgwardt, M. Rasch, B. Schölkopf, and A. J. Smola, “A kernel method for the two-sample-problem,”
Advances in neural information processing systems (NIPS), pp. 513–520, 2007.
May 24, 2017
DRAFT