Answering Opinion
Questions
CSC 594
Jared von Halle
Introduction
•
What makes opinion questions unique?
• More ambiguous
• More open ended
• Can be longer
•
What makes opinion answers unique?
• More variation in possible answers
• No “right” answer
• May be multiple answers
•
What makes opinion questions hard to answer?
• What does it mean to be right?
• How many stances do you present?
• How ridiculous is too ridiculous?
Separating Facts from
Opinions
• Why does this help?
• Compare sentence similarities (e.g. Cosine
Similarity)
• SIMFINDER, a non-hierarchical clustering
technique
• Naïve Bayes
• Unigrams, Bigrams, Trigrams, POS
• Proximity to known positive/negative words using log
likelihood forumal
Obtaining the Training Set
• Somehow, need a set of pre-classified sentences to
train on
• Building by hand
• Inferring from document type. But how do you get
the document type?
• Does an Opinion Document imply Opinion
Sentences???
• Multiple classifiers + Naïve Bayes
Answering Opinion Questions
• TF-IDF
• Aspect Based Opinion Question Answering
• Random Walks on Graphs
TF-IDF
• Term Frequency Inverse Document Frequency
• Term Frequency: How many times the term appears in the
candidate answer
• Document Frequency: How many candidate answers have
the term in it
• Early, rudimentary approach
• Compare words in question to words in candidate
answers
• Nothing specific to opinion questions
• Requires direct word similarities
Aspect Based Opinion
Question Answering
• Specific approach for opinion questions
• Tries to better understand the nuances of the
question
• 5 steps
•
•
•
•
•
Question Analysis
Question Expansion
Quality Filtering
Subjective Sentence Extraction
Answer Grouping
AQA – Question Analysis
Comparative questions have more comparative adjectives and adverbs
Pattern matching to determine targets
AQA – Question Expansion
• Identify “aspects” of the question
• Start by identifying noun phrases
• Compare noun phrases to known corpus of aspects
• Also uses common patterns
• Rates aspects on a scale of 1 to 5
AQA – Question Filtering
• Most models filter on polarity
•
•
•
•
Find answers that match the polarity of the question
Works well for factual questions
Works well for confirming someone’s point
Does not work well for opinions
• AQA looks for answers on both sides of the question
• Question type matters
• AQA focuses more on filtering based on aspects
AQA – Subjective Sentence
Extraction and Answer Grouping
• Filter out objective statements: utilize fact / opinion
splitting
• Weight answers based on how relevant they are to
the question
• Depends heavily on question type
• Yes/No questions
• Reason questions
• General questions
Evaluation and Weaknesses
• Measuring “correct” is hard for opinions
• Evaluation generally requires human evaluation of a
training set
• AQA ignores facts
• Relies heavily on part of speech tagger’s
performance
Precision and Recall
Random Walks on Graphs
• Rejects prior methods involving separate phases and then linear
combination
• Attempt to understand relationship between different answers
• Also attempt to understand relationship between answers and
other characteristics of the question
• Candidate answers are nodes and edges between them
represent similarity measure
• Another set of nodes represent sentiment words
• Connect the answer nodes to the sentiment nodes for a twolayer link graph