Using Visualization To Make
Sense Of Machine Learning
Models
Enrico Bertini
Assistant Professor
Enrico Bertini
My InfoVis Research Group at
NYU Tandon
Josua Krause
Cristian Felix Da Silva
Anshul Pandey
Jay Koven
We study and develop methods to help people
make sense of data (and models) using
interactive data visualization.
Interactive Data Visualization???
Visualization Pipeline
DATA
Data
Processing
PROCESSED
DATA
Ah!
???
Visual
Encoding
VISUAL
REPRESENTA
TION
Perception
Help people think with data …
NYPD Vehicle Collision Data
How can we use the power of interactive
visualization to help people interpret and
validate machine learning models?
DATA
Training Method
INSTANCE
MODEL
PREDICTION
“What kind of decisions
does the model make?”
“Do the decisions
make sense?”
“Where are the
mistakes?”
“How can I fix them?”
INSTANCE
MODEL
PREDICTION
Can we “navigate” the model to create a
“mental model” of how the ML model
behaves?*
* … and hopefully also understand why.
Basic idea: use local instance-level
explanations to inspect the model
globally.
Let’s take a look at two examples we
developed in our lab ...
Both based on evidence counterfactuals:
Original
1
1
0
1
0
0
1
C
T
Explanation
1
0
0
0
0
0
1
C
F
Rivelo (demo)
Explainer (demo)
So … What can VIS do for ML
understanding?
DATA
Mostly covered by
existing methods, except
understanding large set
of features.
Training Method
INSTANCE
MODEL
PREDICTION
DATA
Training Method
INSTANCE
MODEL
Observe the training
process to improve it
(especially with deep
learning).
PREDICTION
DATA
a. Understand model
decisions and structure.
Training Method
b. Compare models.
INSTANCE
MODEL
PREDICTION
Some questions ...
Who are we trying to support? What kind
of problems and questions do they have?
(e.g., Data Scientist, Domain Scientist, Or?)
Can we be more specific?
How does the way we extract and
present information from models affect
human understanding and decision
making?
ML Models Vs. Mental Models
They have (different) “flaws”
How do we use them “synergistically”?
Enrico Bertini
http://enrico.bertini.io
[email protected]
@FILWD
---
Theory of Human-Model Interaction
Model Knowledge Vs. Human Knowledge
Can we even understand models?