Hang Li
Noah’s Ark Lab
Huawei Technologies


Vision: Intelligent Information Assistant
Products



Research Demos




Huawei AppStore Recommendation and Search
Intelligent Help on Huawei Phone
Image Retrieval
Machine Translation
Natural Language Dialogue
Technologies

Deep Learning for Natural Language Processing
Noah Lab’s Research Areas
Speech and Language
SDN
Network
Maintenance
Network
Optimization
and Planning
Speech
Recognition
Intelligent
Telecommunication
Dialogue
Machine
Translation
Networks
Intelligent Mobile Devices
Big Data Analytics
Huawei
Big Data
Telco Big
Data
Recommendation
Search


Vision: Intelligent Information Assistant
Products



Research Demos




Huawei AppStore Recommendation and Search
Intelligent Help on Huawei Phone
Image Retrieval
Machine Translation
Natural Language Dialogue
Technologies

Deep Learning for Natural Language Processing
• 300 million registered users
• 30 million users login every day
• 100 million downloads every day
• Big data: billions of features and billions of
instances
• Accurate recommendation is still
challenging, particularly for long tail and
cold start
• Accurate recommendation model
• Efficient online learning algorithm
Noah’s Ark Lab is responsible for the development of algorithms
• User can ask questions about usage
of Huawei phones features in natural
language
• Mismatch between question and answer
• “How can I access the Internet?” vs “ways
of connecting to the web”
• Learning to rank and learning to match
technologies
• Recognize more than 90% of the questions
• Accuracy of answer is over 92%
• Currently processes 100K questions a day
Noah’s Ark Lab is responsible for the development of system


Vision: Intelligent Information Assistant
Products



Research Demos




Huawei AppStore Recommendation and Search
Intelligent Help on Huawei Phone
Image Retrieval
Machine Translation
Natural Language Dialogue
Technologies

Deep Learning for Natural Language Processing
Research Demo
Research Demo
Research Demo


Vision: Intelligent Information Assistant
Products



Research Demos




Huawei AppStore Recommendation and Search
Intelligent Help on Huawei Phone
Image Retrieval
Machine Translation
Natural Language Dialogue
Technologies

Deep Learning for Natural Language Processing








Recommendation
Question Answering
Speech Recognition
Natural Language Dialogue
Machine Translation
Image Retrieval
Information Extraction
Deep Learning



Multimodal CNN
Neural Responding Machine
Augmented Attention-based Neural Machine
Translation
Multimodal CNN
• One Convolutional Neural
Network represents image
• One Convolutional Neural
Network represents text
• Multi Layer Perceptron
conducts matching
MLP
……
CNN
a
dog
is
catching
a
ball
Experimental Results
• Experiment
– Trained with 30K Flickr data
– Outperforming other state-of-the-art models
Sequence to Sequence Learning with Attention
Mechanism
A
cat
is
sitting
on
the
mat
yt 1
yt
st 1
st
c1
ct 1
ct
cT '
h1
ht 1
ht
hT
y1
…
s1
x1
…
xt 1
xt
一只
猫
坐
在
yT '
…
sT '
…
xT
垫子
上
• Attention mechanism can
make better alignment
between source and target
sentences
Neural Responding Machine
太
羡慕
y1
…
你
了
yt 1
yt
祝
旅行
yT '
…
st 1
st
c1
ct 1
ct
cT '
h1
ht 1
ht
hT
s1
x1
…
xt 1
xt
每年
冬天
都
来
愉快
…
海南
sT '
xT
度假
• Using both local and global
attention mechanisms
c
Experimental Results
• Experiment
– Trained with 4.4 million Weibo data (Chinese)
– 95% of responses are natural, 76% of responses are
appropriate as replies
Message
Response
Occupy Central is finally over
Will Lujiazui (finance district
in Shanghai) be the next?
I want to buy a Samsung phone
Let us support our national
brand.
Neural Machine Translation
A
cat
is
y1
…
s1
sitting
on
yt 1
yt
st 1
st
the
mat
yT '
…
sT '
c1
ct 1
ct
cT '
h1
ht 1
ht
hT
x1
…
xt 1
xt
一只
猫
坐
在
…
xT
那张 垫子 上
• Using context vectors to
avoid over-translation and
under-translation
• Using context gates to
dynamically control the
impact of attention
Experimental Results
• Experiment
– Trained with 1.25 million LDC data (Chinese-English)
33
31.7
32
31
30.1
30.6
30
29
NMT
+Coverage
28.3
+Context
Gate
28
27
26
BLEU
C-E
Translation
有一些恐怖袭击会愈演愈烈。
NMT
There will be some terrorist attacks.
+Both
Some terrorist attacks will become
more and more intense.


Vision: Intelligent Information Assistant
Products:



Research Demos:




Huawei AppStore Recommendation and Search
Intelligent Help on Huawei Phone
Image Retrieval
Machine Translation
Natural Language Dialogue
Advanced technologies in Deep Learning for
Natural Language Processing