Cancer Metastases
Classification in
Histological Whole
Slide Images
Farhad Ghazvinian Zanjani
Sveta Zinger
Peter H.N. de With
Department of Electrical Engineering
Eindhoven University of Technology
Eindhoven, The Netherlands
Method Outline
Tissue region
segmentation
Patch
extraction
Test-time color
augmentation
CNN
CRF
Blob
analysis
DBSCAN
clustering
Class label
Input slide
2
Method
Tissue Region Extraction
 Ignoring the empty zero-filled regions
 Computing a threshold map
• Assigning an Otsu threshold to each pixel by sliding a window
 Clamping the computed threshold map by 10% of Global Otsu threshold
 Using Morphology for removing small isolated objects in the binary image
3
Method
Tissue Region Extraction
Input Color Slide
4
Method
Threshold map
Binary slide
Small isolated
regions removal
Data Sampling

Random patch extraction from slides
•
•
•
•
•
5
Method
Cropped from original full resolution slides
Select patches inside the tissue-region mask
256x256 pixels
Balanced classes
Ignore some marginal patches which contain both classes and
less than 75% of dominant class
Train-time Data Augmentation
 Flipped up/down/left/right
 Rotated 90/180/270 degrees (fast matrix computation)
 Color augmented
• Transforming to HSV color coordinates
• Adding random offset to H, S and V channels
• Brightness (V channel) has been scaled randomly
 On the fly implementation
6
Method
Convolutional Neural Networks

7
Method
Inception v3 vs GoogleNet (v1)
Original inception module [1]
Inception v3 module [1]
ConvNet Model
Accuracy on Camelyon16
GoogleNet [2]
98.4 %
Inception v3
99.5 %
[1] Szegedy, Christian, et al, 2016.
[2] Winner of Camelyon 2016. https://camelyon16.grand-challenge.org/
Convolutional Neural Networks

8
GoogleNet - Inception v3
• Initial parameters have been optimized on ImageNet12 dataset
• Resizing input to 299x299
• Changing the output softmax layer for two-class prediction
• Changing learning rate by monitoring the performance on
validation set
• Weight decay regularization
Method
False Positive Bootstrapping (hard-mining)
 Retraining the network after adding false positive prediction to the training
set
Effects:
• Increasing accuracy about 1% on test set
• Small reduction in recall
ConvNet Model
Inception v3
9
Method
False positive
bootstrapping
Accuracy On
Camelyon16
-
98.7 %

99.5 %
Test-time Color Augmentation
 Two approaches can be devised for tackling with the high color
variations of staining in pathology WSIs
• Normalizing all slides to the color space of a reference [1]
 Training color space variations to the model
10
[1] Bejnordi, B. Ehteshami, et al. 2016.
Test-time Color Augmentation
 Using color deconvolution [1] for finding ROI of absorbing Hematoxylin
and Eosin
 Using stain standardization method for color conversion [2]
RGB
Hematoxylin
Eosin
Deconvolution
Binarized
HSD color space
Template
Histogram
transformation
Converted color
11
[1] Ruifrok, Arnout C., and Dennis A. Johnston, 2001.
[2] Bejnordi, B. Ehteshami, et al. 2016.
DAB
Examples of Color Conversion (negative and positive patches)
Hematoxylin
Eosin
Original
converted
Hematoxylin
Center 1
Center 1
Center 2
Center 2
Center 3
Center 3
Center 4
Center 4
Center 5
Center 5
Negative samples
12
Method
Eosin
Original
Positive samples
converted
Color Conversion Between Samples of 5 Medical Centers
Target
Center 1
Source
Center 1
Center 2
Center 3
Center 4
Center 5
13
Method
Center 2
Center 3
Center 4
Center 5
Test-time Color Augmentation
 Convert the color of test input patch to have a similar distribution to
the training examples of different medical centers
 Select the less uncertain prediction of the network on the coloraugmented set
14
Method
Post processing – Conditional Random Field (CRF)
15
Method
Zoom ROI
GT
PNet( x=tumor) > 0.5
PNet( x=tumor) > 0.9
CRF
Blob Analysis and DBSCAN clustering
 Computing the major axes of a fitted ellipse to the tumor region
 Computing the area of Hematoxylin mask, inside the tumor region
 If slide has been labeled as itc or micro metastases, then check
by DBSCAN clustering
Input slide
16
Method
Detected positive regions
clustered
Result
 Our method on Camelyon17 test set shows 0.87 kappa score
in patient-level
17
Result
Conclusion
 We used a machine learning method, based on convolutional neural
networks
 Inception v3 has been used as a pixel classifier
 False positive bootstrapping improves the prediction performance
 Test-time color augmentation used for decreasing the prediction
uncertainty
 Using Conditional Random Field improves the label assignment
18
Conclusion