Machine
Translation
Introduction to
MT
DanJurafsky
MachineTransla-on
•  Fullyautoma1c
•  Helpinghumantranslators
EnterSourceText:
这 不过 是 一 个 时间 的 问题 .
Transla1onfromStanford’sPhrasal:
Thisisonlyama@erof1me.
DanJurafsky
GoogleTranslate
•  Friedripeplantains:
•  h@p://laylita.com/recetas/2008/02/28/
platanos-maduros-fritos/
DanJurafsky
MachineTransla-on
•  TheStoryoftheStone(“TheDreamoftheRedChamber”)
•  CaoXueqin1792
•  Chinesegloss:Dai-yualoneatbedonthink-of-with-gra1tude
Bao-chai…againlistentowindowoutsidebamboo1pplantain
leafofon,rainsoundsighdrop,clearcoldpenetratecurtain,not
feelingagainfalldowntearscome.
•  Hawkestransla1on:Asshelaytherealone,Dai-yu’sthoughts
turnedtoBao-chai…Thenshelistenedtotheinsistentrustleof
therainonthebamboosandplantainsoutsideherwindow.The
coldnesspenetratedthecurtainsofherbed.Almostwithout
no1cingitshehadbeguntocry.
DanJurafsky
Difficul-esinChinesetoEnglish
transla-on
•  LongChinesesentences:4Englishsentencesto1Chinese
•  Chinesenopronounsorar1cles(Englishthe,a)
•  Chinesehasloca1vepost-posi1ons,Englishpreposi1ons
•  Chinesebedon,windowoutside,Englishonthebed,outsidethewindow
•  Chineserarelymarkstense:
•  Englishas,turnedto,hadbegun,
•  Chinesetou,‘penetrate’->Englishpenetrated
•  Chineserela1veclausesarebeforethenoun,Englisha]er
•  Chinese:[windowoutsidebambooon]rain
•  English:rain[onthebamboooutsidethewindow]
•  Stylis1candculturaldifferences
•  Chinesebamboo1pplaintainleaf->bamboosandplantains
•  Chineserainsoundsighdrop->insistentrustleoftherain
•  Chinesema‘curtain’->curtainsofherbed
DanJurafsky
AlignmentinMachineTransla-on
DanJurafsky
EarlyMTHistory
1946BoothandWeaverdiscussMTinNewYork
1947-48ideaofdic1onary-baseddirecttransla1on
1947WarrenWeaversuggeststransla1onbycomputer
1949Weavermemorandum
1952all18MTresearchersinworldmeetatMIT
1954IBM/GeorgetownDemoRussian-EnglishMT
1955-65lotsoflabstakeupMT
h@p://www.hutchinsweb.me.uk/PPF-TOC.htm
DanJurafsky
1949Weavermemorandum
•  h@p://www.mt-archive.info/Weaver-1949.pdf
•  “Therearecertaininvariantproper1eswhichare…
commontoalllanguages”
•  ‘WhenIlookatanar1cleinRussian,Isay"Thisisreally
wri@eninEnglish,butithasbeencodedinsome
strangesymbols.Iwillnowproceedtodecode.”’
•  “[If]onecansee…Nwordsoneitherside,then,ifNis
largeenough,onecanunambiguouslydecidethe
meaningofthecentralword.”
8
DanJurafsky
TheHistoryofMT:Pessimism
•  1959/1960
•  YehoshuaBar-Hillel“ReportonthestateofMT
inUSandGB”
•  FAHQMTtoohardbecausewewouldhaveto
encodeallofhumanknowledge
•  Insteadweshouldworkoncomputertoolsfor
humantranslators
DanJurafsky
HistoryofMT:FurtherPessimism
TheALPACreport(1966)
•  HeadedbyJohnR.PierceofBellLabs
•  Conclusions:
•  MTdoesn’twork
•  MTafailure:allcurrentMTworkhadtobepost-edited
•  Intelligibilityandinforma1venessworsethanhuman
•  Wedon’tneedMTanyhow
•  AlreadytoomanyhumantranslatorsfromRussian
•  Results:MTresearchsuffered
•  Fundingloss
•  Numberofresearchlabsdeclined
•  Associa1onforMachineTransla1onandComputa1onal
Linguis1csdroppedMTfromitsname
DanJurafsky
MTinthemodernage
•  1975-1985ResurgenceofMTinEuropeandJapan
•  Domain-specificrule-basedsystems
•  1990-present
•  RiseofSta1s1calMachineTransla1on
Machine
Translation
Language
Divergences
DanJurafsky
LanguageSimilari-esandDivergences
•  Typology:
•  thestudyofsystema1ccross-linguis1csimilari1es
anddifferences
•  Whatarethedimensionsalongwhichhuman
languagesvary?
DanJurafsky
Syntac-cVaria-on:BasicWordOrders
Inmanylanguagesonewordorderismorebasic
•  SVO(Subject-Verb-Object)languages
English,German,French,Mandarin
I baked a pizza
•  SOVLanguages
Japanese,Hindi
English:He adores listening to music
Japanese:kare ha ongaku wo kiku no ga daisuki desu
he
music
to listening
•  VSOlanguages
•  Irish,ClassicalArabic,Tagalog
adores
DanJurafsky
Morphology
•  Morpheme:“Minimalmeaningfulunitoflanguage”
Word = Morpheme + Morpheme + Morpheme +…
•  Stems:(baseform,root)
hope+ingàhoping hopàhopping
•  Affixes
•  Prefixes:An1disestablishmentarianism
•  Suffixes:An1disestablishmentarianism
•  Infixes:hingi(borrow)–humingi(borrower)inTagalog
•  Circumfixes:sagen(say)–gesagt(said)inGerman
DanJurafsky
MorphemesperWord
JosephGreenberg.1954.AQuan1ta1veApproachtotheMorphological
TypologyofLanguage.IJAL26:3.
isola1ng
1
1.06
Vietnamese
2
1.68
English
3
2.17
2.55
Yakut Swahili
(Turkic)
synthe1c
4
3.72
West
Greenlandic
(EskimoInuit)
DanJurafsky
Fewmorphemesperword:Cantonese
“Hesaidthiswasthebiggestbuildinginthewholecountry”
Eachwordinthissentencehasonemorpheme(andone
syllable):
keui wa chyuhn gwok
jeui daaih gaan nguk haih li
gaan
he
say entire country most big
bldg house is
this bldg
DanJurafsky
ManyMorphemesperword:Turkish
uygarlaştıramadıklarımızdanmışsınızcasına
uygar+laş+tır+ama+dık+lar+ımız+dan+mış+sınız+casına
Behavingasifyouareamongthosewhomwecouldnotcauseto
becomecivilized
DanJurafsky
WordSegmenta-on
Arewordboundariesmarkedinwri-ng?
•  Somewri1ngsystems:boundariesbetweenwordsnot
marked
•  Chinese,Japanese,Thai
•  Wordsegmenta1onbecomesanimportantpartoftext
normaliza1onforMT
•  Somelanguagestendtohavesentencesthatarequite
long,closertoEnglishparagraphsthansentences:
•  ModernStandardArabic,Chinese
•  Sentencesegmenta1onmaybenecessaryforMTbetween
theselanguagesandlanguageslikeEnglish
DanJurafsky
Inferen-alLoad:
coldvs.hotlanguages
BalthasarBickel.2003.Referen1aldensityindiscourseandsyntac1ctypology.Language79:2,708-36
•  Hotlanguages:
•  Whodidwhattowhomismarkedexplicitly
•  English
•  Coldlanguages:
•  Thehearerhasmore“figuringout”ofwhothevariousactors
inthevariouseventsare
•  Japanese,Chinese
DanJurafsky
Inferen-alLoad:Thebluenounphrases
arenotintheChineseoriginal
飓风丽塔已经减弱为第三级飓风，
RitaweakenedandwasdowngradedtoaCategory3storm;
ø迫近美国德课萨斯州和路易斯安那州，
[Rita/it/thestorm]ismovingclosetoTexasandLouisiana;
当局表示，
theauthori1esannounced;
虽然 ø在登陆前可能再稍微减弱，
although[Rita/it/thestorm]mightweakenagainbeforelanding,
但 ø仍然会非常危险，
[Rita/it/thestorm]iss1llverydangerous;
ø预料 ø会在当地时间星期六凌晨在德州和路易斯安那州之间登陆，
[theauthori1es]predict[Rita/it/thestorm]willarriveattheTexasLouisianaborderonSaturdaymorninglocal1me;
ø直接吹袭休斯敦市东面的主要炼油设施。
[Rita/it/thestorm]willdirectlyhittheoil-refiningindustryeastof
Houston.
DanJurafsky
LexicalDivergences
•  Wordtophrases:
•  Englishcomputer science
•  Frenchinformatique
•  PartofSpeechdivergences
•  EnglishShe likes to sing
•  GermanSie singt gerne [Shesingslikefully]
•  EnglishI’m hungry
•  SpanishTengo hambre
[Ihavehunger]
DanJurafsky
LexicalSpecificityDivergences
•  Gramma1calspecificity
•  Spanish:pluralpronounshavegender(ellos/ellas)
•  English:pluralpronounsnogender(they)
•  Sotransla1ng“they”fromEnglishtoSpanish,needto
figureoutgenderofthereferent!
DanJurafsky
LexicalDivergences:Seman-c
Specificity
Englishbrother Mandaringege(olderbrother),didi(youngerbrother)
Englishwall
GermanWand(inside)Mauer(outside)
Englishfish
Spanishpez(thecreature)pescado(fishasfood)
Cantonese ngau
English
cow
beef
DanJurafsky
PredicateArgumentdivergences
L.Talmy.1985.Lexicaliza1onpa@erns:Seman1cStructureinLexicalForm.
•  English Thebo@lefloatedout.
Spanish
Labotellasalióflotando.
Thebo@leexitedfloa-ng
•  Satellite-framedlanguages:
•  direc1onofmo1onismarkedonthesatellite
•  Crawl out, float off, jump down, walk over to,
run after
•  MostofIndo-European,Hungarian,Finnish,Chinese
•  Verb-framedlanguages:
•  direc1onofmo1onismarkedontheverb
•  Spanish,French,Arabic,Hebrew,Japanese,Tamil,Polynesian,Mayan,
Bantufamilies
DanJurafsky
PredicateArgumentdivergences:
HeadsandArgumentswapping
Dorr,BonnieJ.,"MachineTransla1onDivergences:AFormalDescrip1on
andProposedSolu1on,"Computa1onalLinguis1cs,20:4,597--633
Heads:
Arguments:
English:XswimacrossY
Spanish:Ymegusta
Spanish:XcrucarYnadando
English:IlikeY
English:Iliketoeat
German:DerTerminfälltmirein
German:Ichessegern
English:Iforgetthedate
English:I’dprefervanilla
German:MirwäreVanillelieber
DanJurafsky
Predicate-ArgumentDivergenceCounts
B.Dorretal.2002.DUSTer:AMethodforUnravelingCross-Language
DivergencesforSta1s1calWord-LevelAlignment
Founddivergencesin32%ofsentencesinUNSpanish/EnglishCorpus
PartofSpeech
Xtenerhambre
Yhavehunger
Phrase/Lightverb XdarpuñaladasaZ
XstabZ
98%
83%
Structural
XentrarenY
XenterY
35%
Headsswap
XcruzarYnadando
XswimacrossY
8%
Argumentsswap XgustaraY
YlikesX
6%
Machine
Translation
Three classical
methods for MT
DanJurafsky
3ClassicalmethodsforMT
•  Direct
•  Transfer
•  Interlingua
DanJurafsky
ThreeMTApproaches:Direct,Transfer,
Interlingual
DanJurafsky
DirectTransla-on
• 
• 
• 
• 
Proceedword-by-wordthroughtext
Transla1ngeachword
Nointermediatestructuresexceptmorphology
Knowledgeisintheformof
•  Hugebilingualdic1onary
•  word-to-wordtransla1oninforma1on
•  A]erwordtransla1on,candosimplereordering
•  Adjec1veorderingEnglish->French/Spanish
DanJurafsky
DirectMTDic-onaryentry
DanJurafsky
DirectMT
DanJurafsky
ProblemswithdirectMT
•  German
•  Chinese
DanJurafsky
TheTransferModel
•  Idea:applycontras1veknowledge,i.e.,knowledge
aboutthedifferencebetweentwolanguages
•  Steps:
•  Analysis:Syntac1callyparsesourcelanguage
•  Transfer:Rulestoturnthisparseintoparsefor
targetlanguage
•  Genera-on:Generatetargetsentencefromparse
tree
DanJurafsky
EnglishtoFrench
English:Adjec1veNoun
French:NounAdjec1ve
•  Thisisnotalwaystrue
Routemauvaise‘badroad,badly-pavedroad’
Mauvaiseroute‘wrongroad’
•  Butisareasonablefirstapproxima1on
•  Rule:
DanJurafsky
Transferrules
DanJurafsky
Transferringthegreenwitch….
38
DanJurafsky
Interlingua
•  InsteadofN2setsoftransferrules
•  Usemeaningasarepresenta1onlanguage
1.  Parsesourcesentenceintomeaningrepresenta1on
2.  Generatetargetsentencefrommeaning.
•  Intui1on:UseotherNLPapplica1onstodoMTwork
•  EnglishbooktoSpanish:libroorreservar
•  DisambiguatebookintoconceptsBOOKVOLUMEandRESERVE
•  Need2Nsystems(aparserandgeneratorforeach
language)
DanJurafsky
Interlinguafor
Marydidnotslapthegreenwitch