Talk and Look: Tools for Ambient Linguistic Knowledge
A Project funded by the European Community under the Sixth Framework Programme
for Research and Technological Development (IST-507802)
D7.3 Public Project Presentation (update 2006)
The problem
© TALK Consortium, 2006
2
The problem
New technologies should make life easier.
But ...
our high-tech environment makes ever greater demands
on people
We need …
natural conversational interaction instead of complex
controls and operating instructions
© TALK Consortium, 2006
3
TALK objectives
Natural communication requires
 Content: users say what they want
 Flexibility: users say it the way they want, with no need to
learn the specific commands a device “expects”
 Adaptivity: system adapts to preferences, knowledge, and
ability of user, and to context
 Learning: system uses dialogue strategies learned from
experience, and continues to learn from its interactions with
users
© TALK Consortium, 2006
4
Background of the TALK project
ISU theory: Information State Updates
for dialogue context management
Reconfigurable systems
TALK
Multimodal dialogue
management
Adaptivity, Learning
of dialogue strategies
Task domains: in-car, smart home
© TALK Consortium, 2006
5
TALK consortium
Overall Coordination
Scientific Coordination
© TALK Consortium, 2006
6
User categories
Industrial users
 Developers of dialogue applications who need better
dialogue design and more efficient development
techniques
 Information about industrial user requirements through
feedback by industrial partners in the consortium
End users
 anybody with a car or a home
 Information about end user requirements through
Industrial partners
Market studies (e.g. JDPower & Assoc, Cisco
Systems)
Project-internal wizard-of-oz experiments and
usability tests
© TALK Consortium, 2006
7
Major exploitable results - Technologies
Methods for designing better – flexible and adaptive –
dialogue systems that learn from interactions with users,
based on ISU technology
Methods for rapid and cost-effective deployment of new
dialogue applications through reconfigurable dialogue
systems:
 Separating domain-specific information from generic
communicative behavior
 Separating central aspects of dialogue structure from
modality- and language-specific realisation
© TALK Consortium, 2006
8
Subsidiary exploitable resuls
 Reusable software tools (e.g. TrindiKit, ATK, GF, DIPPER)
 Reusable annotated data archives and databases of
application-specific knowledge, multimodal grammar library
 Design and methodology for conducting multimodal wizardof-oz experiments, user testing of baseline in-car systems
 Contributions to standards (W3C, ISO, standards workshop
with AMI project and W3C 12/12/05)
 Framework and system for ontology-based, generic
presentation planning (DFKI/USAAR In-Car showcase,
PATE/RDFS)
 Skills development: training PhD and Master students
© TALK Consortium, 2006
9
Research results and resources (December 2005)
Corpora (annotated data archive):
 SACTI 1 and 2 (UCAM)
 MIMUS (USEV)
 Sammie 1 and 2 corpora (DFKI/USAAR): MP3 WOZ
corpora annotated at various levels
 ISU-annotated COMMUNICATOR data (UEDIN)
Research results (highlights):
 Multilingual & multimodal grammars (e.g Tram Info and
AgendaTalk dialogue systems, D1.2b)
 The Hidden Information State model (D4.3)
 Reinforcement Learning of dialogue strategies that
outperform all COMMUNICATOR systems (D4.1)
Software:
ATK, Trindikit 4, GF 2.4, Reinforcement Learner, User
Simulations, Automatic ISU annotator, several dialogue
systems
© TALK Consortium, 2006
10
Exploitation strategy
Open source software
 530 TrindiKit downloads, 413 GF downloads, 2985 ATK
downloads, DIPPER 66 downloads
 approx 43,000 hits on TALK website in december 05
Continuous transfer of exploitable results to Industrial partners
Business development executive (UEDIN)
 Funded by Scottish Enterprise (national economic
development agency)
 Full time effort devoted to transfer of research in
speech and language technology from University to
industry (Licensing, Consulting, Joint projects,
Spinouts)
 Similar service by Contact Office for Knowledge and
Technology Transfer (USAAR)
© TALK Consortium, 2006
11
Dissemination Strategy: Three target groups
Research community: via scientific publications
 11 refereed journal papers, 38 refereed conference papers
 18 talks and invited lectures
 Standards workshop: TALK/AMI/W3C December 2005
General public:
 More than 15 press articles
 Press release, internet news releases, TV presentations,
public demonstrations
Industry:
 Uptake by industrial partners, testing by BMW and BOSCH
 Engagement with commercial R&D teams
 Planned presentations at Trade Fairs (e.g. CeBIT)
Training:
 5 PhD (4 completed), 19 MSc (6 completed)
 Anzere Spring School, ESSLLI 2006 advanced lecture
course
© TALK Consortium, 2006
Project website: www.talk-project.org
12
Milestones
Major milestones achieved in 2005:
 In-car systems Version 1.0
 Adaptive dialogue system trained with human data
 International press release
 Standardization workshop with AMI and W3C (12/12/05)
 User testing of the baseline in-car systems
Future milestones:
 Reconfigurable systems (early 2006)
 TALK workshop at a major conference (2006)
 Advanced lecture course at ESSLLI 2006
 Evaluation of learned versus hand-coded systems
 Final evaluation of the in-car showcase (late 2006)
Long-term effects:
 Scientific community will build on TALK results, standards, tools
and resources (as happened with TRINDI and SIRIDUS)
 Industry will use TALK methods and tools for research and
development
© TALK Consortium, 2006
13