ADAPT Undergraduate Internship Programme 2017
PROJECT DESCRIPTION
Institution/Team:
ADAPT/ThemeB
Project Title:
ImageCaptionGeneration
Suitable for students
who are studying in
the following areas:
Skills needed:
Computersciencewithstrongprogrammingskills.
Project Description:
The Role of the
student & benefits
gained from
participation in this
Agoodprogrammingskills.
Familiarwithmachinelearning,computervisionandnaturallanguageprocessing.
Inarecentsceneshift,theSocialMediaerahasthrownupamultitudeoftasksin
whichvisionandlanguageareinherentlylinked.
Mostbusinessesoperatingacrossinternationalbordersunderstandthevalueof
multimodaluser-generatedcontent.Inordertomakeaconnection,theyhaveto
beabletospeakthelanguageoftheircustomersandunderstandtheirneeds.
Websites,marketingmaterials,socialmediaprofilesandotherhigh-impact
elementsindifferentmodalitiesshouldallbethoroughlylocalized,whichcanmean
acombinationofhigh-qualitylanguageprocessing,analyticsandcomputervision.
Inspiredbyrecentworkinmultimodalnaturallanguageprocessingsuchasthe
MultimodalMachineTranslation,thecaptiongenerationmodelsbecomeastrong
tecniquetocaptureanddetermineobjectsintheimagesandexpresstheir
relationshipsinnaturallanguage.
Inthecontextweproposeaninternshipongeneratingsentencesthatdescribea
givenimagecrowledfromSocialMedia.
ThebaselinesystemforthistaskwillbetheimagedescriptionmodelbyXuetal.
(2015).
Thestudentwilllearnhowtouseexistingcaptiongenerationtools(Xuetal.2015)
andimplementsomenewfeaturesthatimprovetheresultsoftheproject.
1
project:
Who will be working
with you?
Short description of
the group:
Recommended
Reading Material:
Other information:
For further details on
this project please
contact:
1
Our undergraduate student will be working closely with Dr. Haithem Afli and Dr.
Jinhua Du. The student will participate in all our project meetings during his/her
timewithus.
TheADAPTThemeBgroup,ledbyProf.AndyWay([email protected]),is
oneoftheleadingMTgroupsonaglobalbasis.
[1]RamiAl-Rfou'etal.Theano:APythonframeworkforfastcomputationof
mathematicalexpressions.CoRRabs/1605.02688(2016)
[2]KelvinXu,JimmyBa,RyanKiros,KyunghyunCho,AaronC.Courville,Ruslan
Salakhutdinov,RichardS.Zemel,YoshuaBengio:Show,AttendandTell:Neural
ImageCaptionGenerationwithVisualAttention.CoRRabs/1502.03044(2015)
[3]MarkMarsdenetal.DublinCityUniversityandPartners’ParticipationintheINS
andVTTTracksatTRECVid2016.
In:TRECVidConference,14-16Nov2016,Gaithersburg,Md.,USA.
Name:
Phone:
E-Mail:
Website:
Dr. Haithem Afli, Dr.JinhuaDu
17006711
[email protected] , [email protected]
Thisisaninitialdescriptionoftheroleofthestudentanditisliabletochangefollowingdiscussionswiththeinvestigators.