Data-DrivenGhostingusingDeepImitationLearning
HoangM.Le1,PeterCarr2,YisongYue1,andPatrickLucey3
CaliforniaInstituteofTechnology1,DisneyResearch2,andSTATSLLC3
1. Introduction
Currentstate-of-the-artsportsstatisticscompareplayersandteamstoleagueaverageperformance.
For example, metrics such as “Wins-above-Replacement” (WAR) in baseball [1], “Expected Point
Value”(EPV)inbasketball[2]and“ExpectedGoalValue”(EGV)insoccer[3]andhockey[4]arenow
commonplaceinperformanceanalysis.Suchmeasuresallowustoanswerthequestion “howdoes
thisplayerorteamcomparetotheleagueaverage?”Even“personalizedmetrics”whichcananswer
howa“player’sorteam’scurrentperformancecomparestoitsexpectedperformance”havebeenused
tobetteranalyzeandimprovepredictionoffutureoutcomes[5].
Thesemeasureshaveenhancedourabilitytoanalyze,compareandvalueperformanceinsport.But
they are inherently limited because they are tied to a discrete outcome of a specific event. For
example,EPVforbasketballfocusesonestimatingtheprobabilityofaplayermakingashotbasedon
thecurrentsituation,andislearntoffenormousamountsofhistoricaldata.Thegeneralusecaseis
thentoaggregatetheseoutcomes,andcompareandrankthemtoseehowvariousplayersandteams
comparetoeachother.Incontrast,whatwe’dreallyliketoknowishowteamscreatetimeandspace
forscoringopportunitiesatthefine-grainlevel.
Withthewidespread(andgrowing)availabilityofplayerandballtrackingdatacomesthepotential
toquantitativelyanalyzeandcomparefine-grainmovementpatterns.Anexcellentexampleofthis
wasthe2013ESPNarticlewrittenbyZachLowe,whichdescribedhowtheTorontoRaptorswere
using“ghosting”toanalyzeplayerdecision-makinginSTATSSportVUtrackingdata[6].Specifically,
theRaptorscreatedsoftwaretopredictwhatadefensiveplayershouldhavedoneinsteadofwhat
theyactuallydid.Developingthecomputerprogramrequiredsubstantialmanualannotation,but
the insight gained turned heads because it made the effectiveness of defensive positioning both a
measurableandviewablequantityforthefirsttime.
Motivatedbytheoriginal“ghosting”work,weshowcaseanautomatic“data-drivenghosting”method
usingadvancedmachinelearningmethodologiesappliedtoaseason’sworthoftrackingdatafroma
recent professional league in soccer. An example of our approach is depicted in Figure 1 which
illustrates a scoring chance that Fulham (red) created against Swansea (blue). Suppose we are
interestedinanalyzingthedefensivemovementsofSwansea.Itmightbeusefultovisualizewhatthe
teamactuallydidcomparedtowhatatypicalteamintheleaguemighthavedone.Usingourapproach,
we are able to generate the defensive motion pattern of the “league average” team, which
interestinglyresultsinasimilarexpectedgoalvalue(69.1%forSwanseaand71.8%forthe“league
average” ghosts -- to fully appreciate the insights revealed by data-driven ghosting, we urge the
readers to view the supplemental video at http://www.disneyresearch.com/publication/datadriven-ghosting.)
2017ResearchPapersCompetition
Presentedby:
1
Figure1Ourdata-drivenghostingmethodcanbeappliedtovariousgamecontextstobetterunderstanddefensivestrategies.
Inthedepictedscenario,Fulham(red)scoresagoalonSwansea(blue).Ghosts(white)representwhereSwanseadefenders
shouldhavebeenaccordingtoaleagueaveragemodel(LAG)andManchesterCitymodel(MUG).
Howeverinpractice,acoachoranalystmaynotwanttocomparetheirdefensetotheleagueaverage,
buttoanotherspecificteam.By“fine-tuning”ourleagueaveragemodeltothetrackingdatafroma
particular team, our data-driven ghosting technique can estimate how each team might have
approached the situation. For example, the coach/analyst may want to see how Manchester City
woulddefendthesameattackingplay.Usingourapproach,wecannowseehowtheywoulddefend
differently(Figure1(right)),andhowmuchitchangestheEGV(69.1%to41.7%).Theotherbenefit
ofusingourghostingapproachisthatissavesthecoach/analystfromsearchingforsimilarplaysin
othermatches(whichmaynotevenexist).
To achieve automatic ghosting, we leverage a machine learning method called “deep imitation
learning”.OurmethodologyresemblestechniquesusedtoteachcomputerstoplayAtari[7]andGo
[8]. We modify standard recurrent neural network training to consider both instantaneous and
futurelosses,whichenablesghostedplayerstoanticipatemovementsoftheirteammatesandthe
opposition.Moreimportantly,ourapproachavoidstheneedforman-yearsofmanualannotation.
Ourghostingmodelcanbetrainedinseveralhours,afterwhichitcanghosteveryplayinreal-time.
Inthenextsections,wedescribethemethodologybehindourghostingsystem,andshowcasehow
automated ghosting can provide insightful analyses and comparisons of team defensive behavior.
Wealsoemphasizethatourapproachisgeneral,andcanbeappliedtoawiderangeofsportssuchas
basketballandfootball.
2.DeepImitationLearningforModelingDefensiveSituations
Whilethemathematicalbackgroundrequiredtoimplementourdeepimitationlearningmethodcan
seemcomplicated(seeAppendixAforfulldetails),thehigh-levelintuitionisquitesimple.Inthis
section we provide an overview of deep imitation learning, and how it can be applied to soccer
trackingdata.
2017ResearchPapersCompetition
Presentedby:
2
Forthispaper,weused100gamesofplayertrackingandeventdatafromarecentprofessionalsoccer
league.Asweareinterestedinmodelingdefensivesituations,weonlyfocusedonsequencesofplay
where the opposition had control of the ball. A defensive sequence is terminated when a goal is
scoredagainstthedefendingteam,theballgetsoutofthepitch,dead-balleventsoccur(e.g.foul,offside),orthedefensiveteamregainspossessionoftheball.Intotal,therewereapproximately17400
sequences of attacking-defending situations (~3 million frames at 10 frames per second). The
averagelengthofallsequencesisapproximately170frames,or17seconds.
2.1.Whatis“DeepImitationLearning?”
Beforeexplainingwhat“deepimitationlearning”is,wefirstexplainwhat“imitationlearning”is.In
manycomplexsituations,itcanbeverychallengingforahumanexperttodescribeandcodifythe
policyorstrategyduetothegranularityorfidelityofthesituation.Forsuchtasks,wecanusemachine
learningtoautomaticallylearnagoodpolicyfromobservedexpertbehavior,alsoknownasimitation
learning or learning from demonstrations, which has proven tremendously useful in control and
roboticsapplications[9-14].
Duetothedynamic,continuousandhighlystrategicnatureofsportslikesoccer,hand-craftingor
manually describing strategy at a fine-grain level is equally problematic. For example in Figure 1,
gettingahumantodescribethelocation,velocityandaccelerationofeveryplayerattheframe-level
wouldbeprohibitivelytime-consuminganderror-riddled.Evenifahumanwereabletodescribethe
playviarules,itishighlyunlikelythatanotherhumanwouldbeabletolearnfromsuchrulesasit
would surely miss some important context or other information. In practice, a human would just
observe many examples until they could understand what to do at a conceptual level. Teaching a
computer is no different. The key is to first obtain the right representation which can enable the
computertolearnfromtheobservations.Inrecentyears,deeplearninghasproventobeapowerful
toolcapableoflearningamulti-layerrepresentationhiddeninthedata,enablingautomaticfeature
discovery that saves tremendous amount of human engineering effort. In this work, we bring
togetherelementsfromautomaticformationdiscovery[16],imitationlearningcombinedwithdeep
learningmethodstolearncomplexrelationshipsfromhigh-dimensionalspatiotempotalsportdata
domainssuchassoccer.
2.2.FormationDiscoveryandDeepImitationLearningApplication
Asthedatacomesfromdifferentteamsandplayers,
onekeycomponentofthepre-processingstepisrolealignment(ororderingtheplayersinaformwhere
the computer can quickly compare strategically
similarplays).Weextractthedominantroleforeach
player from both the defending and attacking team
based on the centroid positions throughout the
segmentofplay,regardlessofthenominalpositionof
such player. For example, a player whose nominal
roleiscentraldefendermayfindhimselfoccupying
thedominantroleofamidfielderincertainsequence
of play. Instead of enforcing a pre-determined
formationontotheteams,thecentroidpositionsfor
2017ResearchPapersCompetition
Presentedby:
3
eachsequenceareautomaticallydiscoveredfromdatabyclusteringeachrole,viaalinearassignment
algorithm, to a role centroid represented by a mixture of Gaussian distributions, in a way that
maximizes the self-consistency within role from one segment of play to another (resembles the
methodfrom[16]).Theresultisanaverageformationacrosstheseasoncloselyresemblinga4-4-2
formation.
As soccer is fundamentally a spatial game, one would expect the geometric relationship among
playersandtheballtocontainimportantsemanticandstrategicvalues.Weformthefullinputvector
toourmodelbyincludingnotonlytheabsolutecoordinatesofplayersandball,butalsotherelative
polarcoordinates(distanceandangle)ofeachplayertowardstheball,goal,andtherolethatwetry
to model. The full feature vector at each time step contains geometric features for all roles in the
formationintheformofonemini-blockforeachrole.Thesemini-blocksarethenstackedinafixed
order consistent with role alignment. In order for the model to identify one-on-one versus zone
coverage, we also indicate which mini-block of input features correspond to the closest three
positionstotherolebeingmodeledateachtimestep.Inaddition,anextrainputvectorindicatingthe
teamidentityisaddedtotheinput.Thisidentityvectorisusefulforlearningparticularstructuraland
stylistic elements associated with different teams, allowing us the study the impact of tuning our
ghostingmodeltodifferentteamstyles.
Weuserecurrentneuralnetworks,apopulardeep-learningtool,tolearnthefine-grainedbehavior
modelforeachroleintheformationineachtimestep.Aparticulartypeofrecurrentneuralnetworks
calledLongShort-TermMemory(LSTM)wasusedduetoitspowerfulabilitytocapturelong-range
dependenciesinsequentialdata.Themodeltakesinasequenceofinputfeaturevectorsasdescribed
above and the corresponding sequence of each player’s positions as output labels. Each player is
modeledbyanLSTM,whichconsistsoftwohiddenlayersofnetworkswith512hiddenunitsineach
layer.Theroleofthesehiddenunitsistocapturetheinformationfromtherecenthistoryofactions
fromallplayersandmaptheinformationtothepositionofthenexttimestep,inamanneranalogous
tohowanAIprogramwastrainedfromdatatomapthehistoryofthegametothenextframeof
actioninAtariandGo.
Usingthestandarddeeplearningapproach,however,provesinsufficienttolearnarobustbehavior
model,duetothetypicallylongtemporalspanofsequencesofplayandsheerhigh-dimensionalityof
thelearningproblem.Intuitively,aswithanyregressionmethod,themodel’spredictionscandeviate
fromthegroundtruthlabels.Insequentialmodelingsettings,thisdeviationcancompoundovertime
andcanleadtoseriousmodelingerrors.Toaddressthisissue,weleveragetechniquesfromimitation
learning. The main idea is that we want to train a model that can learn to recover from its own
prediction mistakes so that the model can be robust over long sequences of decisions. We use an
imitationlearningalgorithmthatlearnstocapturenotonlythebehaviorofeachroleintheteam,but
alsohowmultipleplayersineachteamjointlybehavefromoneframetothenext.Afulldescription
ofourmachinelearningapproachisgivenintheappendix.
2017ResearchPapersCompetition
Presentedby:
4
3.CharacterizationofTheAverageGhostingTeam
Asafirstchecktoseeifourleague-averagemodelpassestheeye-test,wedeploythetrainedmodel
on held out sequences to inspect whether the trained model behaves in a sensible manner. We
observe that our model learned to maintain solid defensive formation and structure, with the
modeledplayersmovinginamannerthatexhibitsspatialandformationalawareness.Toillustrate
this,weshowthreeexamplesofplayinFigure3.
Inthefirstexample,theghostingplayers(white)fromLiverpoolmovetogetherwiththerestofthe
teammates (blue) in pressing higher up the pitch in a situation when Manchester City (red) just
regained possession in the middle of the field. To avoid clutter, we display only the ghosting
trajectories(inwhite)ofthefiveleftpositionsoftheteam.
Figure3Examplesofghostingbehaviorforour“leagueaverage”model.
Because our algorithm models the interactions between teammates, our ghosted players exhibit
high-level coherent team behaviors. In the middle panel of Figure 3, ghost player number 5 of
Sunderlandbrokefromformationtochallengetheballcarriernumber7ofAstonVilla(red).Ghost
player number 9 of Sunderland swaps roles with ghost number 5 and drops back to mark the
attackingnumber6.
Inmanysituations,thebehaviorofthe“leagueaverage”teammaybesubstantiallydifferentfromthe
actualplaysuchthattheoutcomecouldbedifferent.IntherightpaneofFigure3,theghostingplayer
number8moreproactivelyclosedthepassinglanethatcouldhavepreventedtheshotongoalby
attackingnumber6(red),whichhappenedinrealityduetothefactthatbothnumber5and8from
QPR(blue)dropmoredeeplyandyieldedopenspace.
3.1VariancefromTheAverageGhostingbyPositionsandTeam
We can quantify how the players and teams differ from the “average” team. The average level of
deviationacrossallplayersandteamsisabout~4meters.Toputthisnumberincontext,notethat
2017ResearchPapersCompetition
Presentedby:
5
thisisahighlyaccurateaveragelevelofprecision,asthemodelhastotakeintoaccountthearbitrarily
longsequenceofplay.Incontrast,morenaivemachinelearningapproaches(thatdonotaccountfor
errorpropagationthroughtime)wouldsufferfromveryhighlevelsofpredictionerror(frequently
inthe20-30metersrange).Wecanfurtherbreakdownthedeviationintogroupsofposition.Forthe
defenderpositions,themajorityofthedeviationsarelessthan3metersapartfromtheactualplayer
positions.Thedeviationincreasesasthedominantpositionsmovefurtherupthefield.Thisreflects
increasinglevelofvariationinhowattack-orientedplayerswoulddefend.Thedefensivebehaviorof
a striker, for example, could change significantly from team to team and among specific players,
leading to a higher level of variance by the “average” ghosting positions. We show this deviation
accordingtopositioninFigure4.
Figure4:Deviationfromtheaverageghostingmodelbypositions
(Central Defenders (green), Central Midfielders (blue) and
Forwards(red)
Forourteamanalysis,weusedall20teamsinour
dataset. As we can not disclose the specific
performance of these teams, we denote them as
teams A-T. At the team level, this variance can
indicate which team differs the most from average
behavior in terms of defensive positioning. We
quantifythisoutofpositionratiobyusingan80-20
rule:aplayeratanygivenmomentisconsideredout
ofpositionifhisdeviationfromhisghostisinexcess
ofthe80thpercentileoftheentireleagueaverage.A
breakdown of this tendency gives us a sense of
whichteamsusenon-conventionalformations.InFigure5,theteamsaresortedbythetotalnumber
ofgoalsconcededovertheseasoninanincreasingorder.Weanalyzedeachteam’sbehaviorviatwo
groups: i) “back line” positions (backs), and ii) “front line” positions (midfielders and forwards).
While this positioning variance is only one part of the whole picture, note that both the top and
bottomteamsintheleagueintermsofthegoalconcededwerethetwooutliers.Forthetopteam,the
deviation may come in part from the attackoriented positions (wingers and forwards)
exhibiting a wider range of movement relative to
other teams. In the case of the bottom team, who
ended the season with a relegation, both
dominantlydefensiveandoffensiveplayerstendto
driftfarfromtheleagueaverage.
Figure 5: The out of position frequency (large deviations) by
teamforboththebackandforwardpositions.Teamsareordered
from left to right sorted by the overall goals conceded in the
entireseason
2017ResearchPapersCompetition
Presentedby:
6
3.2ExpectedGoalValueofGhostingTeam
Anaturalusecaseforghostingistostudyhowhypotheticalscenariosmayunfold,i.e.,counterfactual
reasoning.Inadditiontoqualitativeassessmentsofghosting,wealsowishtoquantitativelyassess
theeffectofalternativedefensivereactionstothesamesituation.Asacasestudy,weanalyzedsafetycriticalplaysequences,suchasgoalscoringopportunities,toseehowtheplaymaybealteredvia
ghostingtoimprovethedefensivepositioning.
To analyze the goal scoring opportunities, we extracted shot events from the entire season.
Concretely,wefocussedonthe10-secondsegmentsleadinguptoanopen-playshotevent,eitheron
orofftarget(wedidnotusepenalties,free-kicks,setpieces,andgamesequenceswithlessthan
11playersfromeitherside).WequantifiedtheperformanceofeachmodelusingtheExpectedGoal
Value(EGV)metric[3],whichcanestimatethegoalscoringprobabilityofeachshotbasedonthe
recentplayerandballpositionsandevents.SimilartothesetupinFigure1,foreachshotsequence,
theaverageghostisinitializedbythecurrentpositionsofthedefensiveplayersonlyfortheveryfirst
frame,andthesequenceisunfoldedthereafterbyrunningtheghostingmodelacrosstheremaining
timesteps.Notethatallotherfactorsofthegivensequenceremainedunchanged(attackingplayer
positionsandballmovement).Attheendofthesequence,theexpectedgoalprobabilityiscalculated
basedonthecollectivebehavioroftheghostingteam.TheoverallEGVisthesumofexpectedgoal
valuesoverallsequences.
Theexpectedperformanceoftheleagueaverageteamversusthe
actualoutcomesareshowninTable1.TheEGVfromopen-plays
shot events improved compared to the total numbers of goals
concededfromall20teams(EGVof474onactualgoalcountof
494fromopen-plays),primarilyduetothe“leagueaverage”team
abletolowerthescoringchanceagainstseveraloftheweakest
teamsintheleague(teamI,J,K,LandT).
4.QuantifyingtheEffectofTeamStyle
Analystsandcoachesoftenwanttocompareteamperformance
withnotjusttheleagueaverage,butalsowithspecificteamsand
certain characteristics (attack-oriented, possession-based etc.).
Doingsoatafine-grainedlevel,however,isdifficultandnearly
impossiblewithdiscretestatistics.Ourghostingsystemprovides
awaytonotonlymodeltheexpectedtrajectoryofeachplayer,
but also incorporate the ability to impose stylistic elements of
specific teams in order to answer the question: how would
differentteamsreacttothesamesituation?
Toaddressthisquestion,weemployedtechniquesfrommachine
learning for domain adaptation. Intuitively, by taking an extra
vectorindicatingtheidentityoftheteamasinputtothemodel,
ourdeep-learningbasedmodelcanextractelements
2017ResearchPapersCompetition
Presentedby:
7
relevanttoeachteam’splayingstyle(suchasspatialarrangement,aggressivenessetc.).Inanygiven
ghostingscenario,the“average”teammodelcanthenbeadaptedtotheplayingstyleofanyteamin
theleaguebychangingtheteamidentityvector,thusallowssimulatinghowtheghostteamplaying
withadifferentteamstylewouldfareunderthesamescenarios.Thisisanalogoustorecentdeep
learningadvancesinstyletransfer,wherethestylisticelementsfrompaintingsandpicturescanbe
extractedwithadatasetconsistingof,forexample,VanGogh’sworks,sothatVanGogh’spainting
stylecanbetransferredtootherimages[15].
Westudyhowdifferentstylescanimpactthedefensiveperformance,comparedtotheaveragemodel.
With domain adaptation, the average model can take on the identities of each of 20 teams in the
league,onall6020open-playshoteventsacrosstheentireseason.Wethencomparehowtheaverage
ghostingteamandteam-specificghostsperformrelativetotheactualoutcomes,andtoeachother.
Figure6Theeffectofassigningdifferentteamstylestoaverageghostingmodelintermsoftotalexpectedgoalvaluesover
season’sopen-playshots(green)vs.totalactualnumberofgoalsconcededbyeachteam(blue)
OurresultsaresummarizedinFigure6.Interestingly,noticethedifferenceindefensiveperformance
of each team style. Since all other factors are controlled, the difference in overall expected goals
concededcanbeattributedtodifferentdefendingstyles.Weobservea61.8%correlationbetween
the total EGV coming from different ghosting styles with the overall number of goals conceded in
reality.Whileluckandindividualskillsmatterforanyteam,teamF’sdefensiveperformanceseems
attributable to an efficient defensive formation (4-5-1 pressing system under a new coach. The
formationdiscoverymethodofSection2.2appliedtoteamFalsoindicatestheyfrequentlyplaywith
5midfielders,withthevarianceofmidfielders’movementshigherthantherestoftheleague).During
theactualseason,teamFalsopossessedthesecond-lowestgoalconcededperopen-playopportunity
intheleague(calculatedfromTable1),astatisticconsistentwiththeefficiencyofteamF’sghost.Of
course,thisisonlyonepieceofthedefensiveequation,sinceshoteventsinisolationdonotreflect
howteamsorganizetheirdefenselongpriortotheshotopportunity.Inaddition,EGVforthistype
ofsituationisnotperfect,sinceitdoesnotcapturethepossibilityofdefensiveghoststerminatingthe
shot opportunity altogether. Nonetheless, ghosting to different teams in this manner is a still an
interestingandnovelwaytocomparedefensivebehaviorsacrossteamsonanequalfooting,without
relyingondisparatestatisticscomingfromdifferentgamesandscenarios.
2017ResearchPapersCompetition
Presentedby:
8
The approach we described thus far is a data-efficient way to extract “personalized” strategic
elements from different teams to facilitate both modeling and evaluation. We emphasize that our
approachisgeneralandcanbeappliedtomodelingnotjusttheaverageteamintheleague,butalso
team-specificmodels(assumingsufficienttrackingdata).
5.ExampleofTheDynamicsofGhosting
Nowthatwehaveabetterideaofhowourghostingmethodworks,andhowthe“leagueaverage”
modelvariestoaspecificteam’smodel,wecanrevisittheexampleshowninFigure1andbreakdownthisplayingreaterdetail(aseparatevideohighlightingmanydifferentscenariosisavailable
athttp://www.disneyresearch.com/publication/data-driven-ghosting).
Asbefore,weconsiderthesequenceofplaybetweenFulham(Red,attackingfromleft)andSwansea
(Blue,defendingonright).WecomparewhatSwanseaactuallydidwiththatoftheleagueaverage
ghosts(LAG-white,top)andtheManchesterCityghosts(MCG-white,bottom).Theballtrajectories
areinyellow.Notethatwedonotmodelgoal-keeperpositions,whicharehighlighted(number1).
Toavoidclutter,wesuppressirrelevanttrajectoriestohighlightkeyplayersanddynamics.
Figure7Exampleofthedynamicsofghostingacrossaplaysequencebytwodifferentghostingstyles.
2017ResearchPapersCompetition
Presentedby:
9
TheactualscenarioresultedinagoalscoredagainstSwanseaafterareboundfromacounter-attack
byFulham.Asmentionedintheprevioussection,bothghostingmodelswereinitializedwiththe
actualpositionsofSwanseaplayersinthefirstframe.Theghostingsystemtakesoverandmakes
decisioninreal-timeabouthoweachplayershouldpositionhimselfforeveryframethereafter.
Red#6 and Red#7 open the counter-attack with a “give-and-go pass” to get behind Blue#4 and
Blue#7.Instage1(leftcolumn),bothLAGandMCGbehavesimilarlyastheplayunfolds,andclosely
resembletheactualSwanseadefense.TheonlyexceptionisLAG#8.ComparedtoBlue#8and
MCG#8, LAG#8 proactively challenges the dribble by Red#6, who in reality drew in Blue#5 and
createdawide-openpasstoRed#10,leadingintostage2(middlecolumn).Thisisthekeymoment
in the play. In the actual sequence, Red#10 is left unmarked and gets an uncontested shot on
goalkeeperBlue#1.HerebothMCG#5andLAG#5positionedthemselvesfurtherbackandwereable
togettoRed#10intimetocontesttheshot.Suchattempthypotheticallycouldhavepreventedthe
reboundthatledtothegoal.Asthesequenceofeventsunfoldtostage3,thereboundfoundRed#9,
whoisleftuncoveredbyactualBlue#9,leadingtothegoal(rightcolumn).Thenuanceddifferencein
positioningbyMCG#2andLAG#2endedupmakingamajordifferenceinthescoringchance.Similar
toBlue#2,LAG#2failedtocoverRed#9.MCG#2,however,rushedbackdeeperintothebacklinefrom
earliermomentsoftheattackandwasinapositiontocontesttheshotontheopennet,andcould
have covered for an attempt by both Red#7 and Red#9. The end result was a reduction in goal
probabilityfrom~70%(bothactualsequenceandLAGsequence)to~40%byMCG.
Thefine-grainedsimulationandevaluationofdefensivescenariospresentedherewouldnothave
beenpossibleusingonlydiscretestatisticsaspopularlyusedinsportanalytics.Ourmethodisalso
general and has a rich potential to enable team-specific modeling for both coaching and media
analysispurposes.
6.Summary
Theongoingexplosionoftrackingdatahasnowmadeitpossibletoapplypowerfulmodernmachine
learning techniques to build increasingly fine-grained models of player and team behavior. With
data-driven ghosting, we can now, for the first time, scalably quantify, analyze and compare finegraineddefensivebehavior.Inthispaper,wehavedemonstratedthevalueofourapproachinarange
ofcasestudies.Weemphasizethatourapproachisalsoapplicabletosportsbeyondsoccer,suchas
basketballandfootball.
2017ResearchPapersCompetition
Presentedby:
10
References
[1]http://www.fangraphs.com/library/misc/war/
[2]D.Cervone,A.D’Amour,L.BornnandK.Goldsberry,“POINTWISE:PredictingPointsandValuing
DecisionsinRealTimewithNBAOpticalTrackingData”atMITSSAC,2014.
[3]P.Lucey,A.Bialkowski,M.Monfort,P.CarrandI.Matthews,“QualityvsQuantity:ImprovedShot
PredictioninSoccerusingStrategicFeaturesfromSpatiotemporalData”,inMITSSAC,2015.
[4]B.Macdonald,“AnExpectedGoalsModelforEvaluatingNHLTeamsandPlayers”,inMITSSAC,
2012.
[5]X.Wei,P.Lucey,S.Morgan,M.ReidandS.Sridharan,““TheThinEdgeoftheWedge”:Accurately
PredictingShotOutcomesinTennisusingStyleandContextPriors”,inMITSSAC,2016.
[6]Z.Lowe.“Lights,Cameras,Revolution”.Grantland.19Mar2013.
[7]R.McMillan.“Google’sAIisNowSmartEnoughtoPlayAtariLikethePros”.WIRED.25Feb2015.
[8]C.Metz.“WhattheAIBehindAlphaGoCanTeachUsAboutBeingHuman”.WIRED.19May
2016.
[9]P.Abbeel,andA.Ng.“Apprenticeshiplearningviainversereinforcementlearning.”In
InternationalConferenceonMachineLearning(ICML),2004.
[10]H.Le,A.Kang,Y.YueandP.Carr,“SmoothImitationLearningforOnlineSequencePrediction”.
InInternationalConferenceonMachineLearning(ICML),2016.
[11]B.Argall,S.Chernova,M.Veloso,andB.Browning.“Asurveyofrobotlearningfrom
demonstration”.Roboticsandautonomoussystems,57(5):469–483,2009.
[12]S.RossandD.Bagnell.“Efficientreductionsforimitationlearning”.InConferenceonArtificial
IntelligenceandStatistics(AISTATS),2010.
[13]S.Ross,G.GordonandA.Bagnell.“Areductionofimitationlearningandstructuredprediction
tono-regretonlinelearning”.InConferenceonArtificialIntelligenceandStatistics(AISTATS),2011.
[14]A.Jain,B.WojcikandT.Joachims,andA.Saxena.“Learningtrajectorypreferencesfor
manipulatorsviaiterativeimprovement”.InNeuralInformationProcessingSystems(NIPS),2013.
[15]L.Gatys,A.EckerandM.Bethge.“Aneuralalgorithmofartisticstyle”.InNeuralInformation
ProcessingSystems(NIPS),2015
[16]A.Bialjowski,P.Lucey,P.Carr,Y.Yue,S.SridharanandI.Matthews.“Large-scaleanalysisof
soccermatchesusingspatiotemporaltrackingdata”.In2014IEEEInternationalConferenceonData
Mining(ICDM),2014
2017ResearchPapersCompetition
Presentedby:
11
Appendix
DataPreparation
Duetothelackofhighqualityannotateddata,wefocusedonathirdoftheseasonworthofmatches
(~100games).Forthepurposeofmodelingsoccerdefense,wefurthersegmentthegameevents
intopossessionsequences.Ateamisdefinedtobeinadefensivesituationwhenitisnotincontrol
oftheball.Adefensivesequenceisterminatedwhenagoalisscoredagainstthedefendingteam,
theballgetsoutofthepitch,dead-balleventsoccur(e.g.foul,off-side),orthedefensiveteam
regainspossessionoftheballafterhaving2consecutivetouches.Thispre-processingstepresults
inapproximately17400sequencesofattacking-defendingsituations(~3millionframesat10
framespersecond).
Onekeycomponentofthepre-processingofthedataisrole-alignment.Role-alignmentisnecessary
forreducingthedimensionalityofthelearningproblem,essentiallyprovidingadditionalcontextto
imposeorderingonthetraininginput.Atthesimplestlevel,onecanviewthelearningproblemasa
mappingfromthepreviouspositionsof22playersandtheball,tothepositionofaparticularplayer
atthecurrenttimestep.Onekeyissuewithlearningthiskindofmappingwithoutimposingan
orderingisthatthedatarequirementtolearna“permutation-invariant”behaviorwillneedto
increasebyafactorofbillions((10!)2specifically).Toreducethisdataburden,wefirstextractthe
dominantroleforeachplayerfromboththedefendingandattackingteambasedonthecentroid
positionthroughoutthesegmentofplay,regardlessofthenominalpositionofsuchplayer.Assuch,
aplayerwhosenominalroleiscentraldefendermayfindhimselfoccupythedominantroleofa
midfielderincertainsequenceofplay.Theleagueaverageapproximatesa4-4-2formation.Ineach
segmentofpossession,weorderthetrainingdatabasedonthedominantroles,wherethe
assignmentofroleattemptstomatchthisaverage4-4-2formation,usingtheHungarianalgorithm
(forminimumcostassignment,wherecostmeasurethedistanceofeachroletoeachofthe10
possibleGaussiandistributionsofspatialcoveragelearnedfromdata).
Figure8Statefeaturevectorextractedfromanorderedlistofplayerpositionateachtimestep
2017ResearchPapersCompetition
Presentedby:
12
Assoccerisaspatialgame,onewouldexpectthegeometricrelationshipamongplayersandtheball
tocontainimportantsemanticandstrategicvalues.Weformthefullinputvectorforthepurposeof
trainingbyincludingnotonlytheabsolutecoordinatesofplayersandball,butalsotherelative
polarcoordinatesofeachplayertowardstheball,goal,andtherolethatwetrytomodel.Figure1
describesthestructureofafullinputvectorateachtimestep.Heretherolebeingmodelistheleft
backposition.Thefullinputfeaturevectorrepresentingtheleftbackpositionconsistofmini-blocks
offeaturesforeachrolefromthedefendingandattackingteam,sortedbyafixedorder.Theminiblockforthegoal-keeper,forexample,containsabsolutepositionandvelocity,thedistanceand
anglerelativetotheleft-backposition,thedistanceandangleofthegoal-keepertothedefending
goal,andthedistanceandangletotheballatthecurrenttimestep.Inadditiontostackingtheminiblocksoffeaturestogether,wealsoduplicatethefeaturesfromthemini-blockscorrespondingto
theclosest3positionstotheleft-backateachtimesteps.Thisresultsinafullinputfeatureof
dimension399foreachroleateachframe.Wecallthisinputvectorthestatefeaturevectorforeach
role.
DeepMulti-agentImitationLearning
Theimitationlearningtaskistomapthestatefeaturevectorateachtimesteptothecorresponding
actionoftheplayerbeingmodel,whereactionisdefinedastheplayerpositionatthefollowingtime
steps.Inprinciple,thisisanonlinesequencepredictionproblem,wherethemodeloutputsactionof
aplayerconditionedonthestateofsuchplayer,asrepresentedbytherecenthistoryofactionsof
themodeledplayer,aswellasotherplayers.Anaturalcandidatetoaddresssuchonlinesequence
predictionproblemisrecurrentneuralnetworks(RNN).AparticularlypopularclassofRNNs,Long
Short-TermMemory(LSTM)hasbeensuccessfullyusedinrecentdeeplearningapplicationssuch
asmachinetranslation,speechrecognition,handwritingsynthesis,etc.Twomajordifferences
betweenoursettingandpreviousapplicationsofLSTMare(i)playersneedtomakedecisionin
real-time,renderingthepopularencoder-decoderapproachtosequence-to-sequencemodeling
impractical(ii)ourlearningset-upbelongstotheclassofdynamicalsystemlearningwhereaction
couldaltersubsequentstatedistribution,potentiallycausingamismatchbetweentrainingand
inferencethatcouldseverelylimittheperformanceoftraditionalLSTMmodels.
Figure9Exampleofphase1ofalgorithmwithk=2
2017ResearchPapersCompetition
Presentedby:
13
Ourproposedmethodjointlycombinestrainingandinferencetoaddressbothoftheseissues.This
methodsimulatesonlinepredictioninanofflinefashion,whileallowingthemodeltogradually
learnlonger-rangeprediction.Phase1ofthealgorithmlearnsamodelforeachofthe10rolesofthe
“average”defendingteam(seefigure9foranillustration).Inphase2,weusedthesepre-trained
modelslearnedfromphase1toscaleupthetrainingofsingleplayerintojointtrainingofmultiple
playerstomodelcollaborativemulti-agentlearning(figure10showcasesthejointtrainingof2
players).
DeepImitationLearningAlgorithm:
Todescribeourlearningalgorithminmoredetails,wegenericallydenoteasequenceofstate-action
pairsas{(s0,a0),(s1,a1),…,(sT,aT)}.WeuseT=50inforourghostingapplicationtosoccerdefense.
Phase1:Learnsingleplayermodelforeachrolejin{1,2,..,10}
o Initializearecurrentnetworkmodelgiventheground-truthdataset{(s0,a0),
(s1,a1),…,(sT,aT)}forafewiterations
o Fork=1,2,…,T:
▪ Fort=0,k,2k,..,T:
● Fori=0,1,..,k-1:
o Applythemodeltost+itoobtainactiona’t+i
o Usetheactiona’t+itoupdatethenextstatest+i+1(similarto
structureinfigure1withthenewactiona’t+i)
● Fori=0,1,…,k-1:
o Usethelossbetweenpredictiona’t+iandgroundtruthactionat+i
toupdatetherecurrentnetworkmodelusingstochasticgradient
descent
Phase2:Learnmultiplayermodelsimultaneouslyforallrolejin{1,2,..,10}
o Fork=1,2,…,T:
▪ Fort=0,k,2k,…,T:
● Fori=0,1,…,k-1:
o Applythepreviouslytrainedmodelsforeachrolejtostate
vectors(j)t+itoobtainactiona’(j)t+i
o Usingpredictedactiona’(j)t+itoupdatestatefeaturevectorforthe
nexttimesteps(j)t+i+1forrolejands(j’)t+i+1forallotherrolej’
● Fori=0,1,…,k-1:
o Usethelossbetweenpredictiona’(j)t+iandgroundtruthaction
a(j)t+itoupdatetherecurrentnetworkmodelforrolejusing
stochasticgradientdescent
2017ResearchPapersCompetition
Presentedby:
14
Figure10Illustrationofphase2ofalgorithmwith2playersandk=1
2017ResearchPapersCompetition
Presentedby:
15