Learning to Automatically
Solve Algebra Word Problems
Nate Kushman, Yav Artzi, Luke Zettlemoyer, Regina
Barzilay
钱炘祺
2015.9.14
TASK
map a given algebra
word problem
to a set of equations
representing its
algebraic meaning,
which are then
solved to get the
problem’s answer.
Main Idea
• Select a template to define the overall structure of
the equation system.
• instantiate the template with numbers and nouns
from the text.
Main Idea
• Probabilistic Model
• discriminate between competing analyses using a loglinear model.
• The probability of a derivation y given a problem x:
• find the most likely answer a given a problem x:
Learning
• Induce the structure of system templates in 𝒯
• Estimate the model parameters 𝜃
• use L-BFGS to optimize the parameters. gradient of the
individual parameter
is given by:
• Beam Search
Model Details
• Template Canonicalization
• syntactically different but semantically equivalent.
• “John is 3 years older than Bill”
• j=b+3
j − 3 = b.
• Slot Signature
• different slots may share many of the features used in
deciding their alignment.
• indicates the system of equations it appears in, the
specific equation it is in, and the terms of the equation it
is a part of.
Model Details
• Features
• Document level features
• words in NL indicate the template.
• Single Slot Features
• indicate the queried quantities.
• Slot Pair Features
• relationship between slot words.
• Solution Features
• the final answer --- reasonable.
Experimental
• Dataset
• collect from Algebra.com
• heuristically filter the data to get only linear algebra
questions
• randomly choose a set of 1024 questions.
• clean the data
• remove the infrequent problem types.
Experiment
• Dataset
• collect from Algebra.com
• heuristically filter the data to get only linear algebra
questions
• randomly choose a set of 1024 questions.
• clean the data
• remove the infrequent problem types.
Experiment
• Accuracy for various forms of Supervision
• baseline scenario.
• semi-supervised scenario.
• fully supervised scenario.
5EQ
5EQ+ANS
ALLEQ
Experiment
• performance on different template frequencies
• equation accuracy
• answer accuracy
Experiment
• feature analysis
Error Analysis
• more background or world knowledge might have
helped. 1/4
• compositional language.
1/2
Conclusion
• automatically learning to solve algebra word
problems.
• construct systems of equations, while aligning their
variables and numbers to the problem text.
• use a newly gathered corpus.
• various forms of weak supervision on performance.
• present the first learning result for this task.
Future Work
• develop techniques to learn compositional models
of meaning for generating new equations.
• extend the approach to related domains.
• geometry, physics, and chemistry.
• extend the techniques to synthesize even more
complex structures
• computer programs, from natural language.