Online Fashion Shopping Photos
Poshmark Partners:
Aesthetic
Quality
Assessment
Gautam Gowala
Sathya Sundaram
Professor:
Jan.P.Allebach
Graduate Mentor:
Zhi Li
Student:
Yin Wang
Student:
Tenglun Tan
Department of Electrical and Computer Engineering
Project Pipeline and outlines
Background
Photo OCR Design
 Photo OCR Pipeline:
Poshmark is a mobile and online marketplace for
women's fashion based in Menlo Park, California.
When customer are trying to sell things on
Website, they have to fill out a long list including
Category, Size, Brand, Color, New with tags,
Original Price. We are trying to train a system that
can detect the Category, Color and Brand using
Image Processing and Deep Learning.
 Step1: Text detection
Label the training set with positive examples and negative
examples.
 Color Naming Pipeline:
Positive examples (y=1)
Negative examples (y=0)
Then we use the sliding window detection to find the
location of the text.
Objectives
 Software: Python 2.7 and OpenCV 3.0.
Expansion
❑Deep Learning
 Deep Learning Method: Convolutional Neural Network(CNN).
 Brand Recognition: Photo OCR
Background Color Extraction
 Step 2: Character Segmentation:
❑Image Processing
 Color Recognition: K-Means Algorithm
 Background Removing
Gold, Red
❑Color Naming
 Collect and Categorize Colors from Online Retailers
 Matlab Approach:
• Read photos from the designated path in Matlab
• Approximate the range of RGB value of the item
you want to extract
• Input desired range of RGE value
• Set other pixels as pure white (256,256,256)
• Output would be the item after extracted
 Result:
 OpenCV Approach (C++ source code)
• Read photos from the designated path
• Input different layers on the photos you wan to extract
(usually 4 or 5)
• Do not need to use color space to define to extract the item
• The main item you want would be kept in the photo
 Result:
 Step 3: Character Recognition::
Add background to different font
Psychophysical Experiment
RGB Color Space:
Based on tristimulus theory of human vision
R: red
G: green
Add distortion, noise
B: blue
Original photo
LCH Color Space
After defining layers (4 layers)
After extraction
Color Recognition Design
The Lch color model is very useful for retouching images in a
color managed workflow, using high-end editing applications.
Lch is device-independent.
 Color Recognition: K-Means Algorithm
•K-Means clustering is a method of vector quantization, originally from signal processing, that is popular for cluster
analyzing data mining.
 Use Python2.7 and OpenCV3.0 to extract color from photo using K-Means
•Application: Color Recognition of an item in the photo.
•Demo: We first use Yin’s tool to remove the background, then apply Tenglun’s K-Means Algorithm to get the histogram of the
distribution of color of the item in the photo.
L: lightness
C: Chroma
H: Hue
Systematic Study of Color Categories
Downloading Pictures from 20 On-line Retailers
Asos, Nordstrom, JCPenny, Walmart…
Categorize according to different Colors
Most Popular Colors: Black, Red Pink, White…




After MATLAB extraction
Original photo
Brown, beige,
dark red
Color Frequency Distribution From 20 On-line Retailers
1
19 19
18
19 19
17
0.9
18
19
18 18
18
16
0.7
14
11
0.6
11
12
10
9
0.5
10
8
0.4
8
0.3
5
4
0.2
0
0
0
0
0
0
POSTER TEMPLATE BY:
www.PosterPresentations.com
0
 Application: Artificial Intelligence.
When customer upload an item to sell, he/she will get
the information of Brand Recognition(using our deep
learning approach) and Color Recognition(using our
image processing approach).
Background Removing: Since the pattern is already
extracted from the photo, we can ask the customer
whether they want to remove the background to improve
the quality of photo.
20
18
0.8
0.1
Conclusion
0
0
0
0
0
0
0
0
2
2
0
0
0
0
0
4
0
6
4
1
0
1
0
1
0
2
2
0
0
1
0
0
1
0
0
1
0
1
0
2
0
1
0
2
0
1
0
2
0
0
3
1
0
1
0
1
0
1
0
0
4
1
0
1
0
1
0
1
0
2
0
Red, dark
green, beige
 Future Plan: Apply Ceiling Analysis to figure out
what part of Pipeline to work on Next.
Ceiling Analysis: Estimating the errors due to each
component.