LinkedUp: Linking Web Data for Education Project –
Open Challenge in Web-scale Data Integration
http://linkedup-project.eu/
Coordination and Support Action (CSA)
Grant Agreement No: 317620
D1.2
Challenge Design
Deliverable Coordinator:
Mike Lauruhn, Eelco Herder
Deliverable Coordinating Institution:
Elsevier
Other Authors:
Stefan Dietze
Sander van der Waal
Elisabetta Parodi
Hendrik Drachsler
Mathieu D‘Aquin
Document Identifier:
LinkedUp/2013/D1.2
Date due:
30 April 2013
Class Deliverable:
LinkedUp 317620
Submission date:
30 April 2013
Project start date:
November 1, 2012
Version:
v1.0
Project duration:
2 years
State:
Final
Distribution:
Public
© Copyright lies with the respective authors and their institutions.
Page 2 of 26
LinkedUp Support Action – 317620
LinkedUp Consortium
This document is part of the LinkedUp Support Action funded by the ICT Programme of the
Commission of the European Communities by the grant number 317620. The following partners are
involved in the project:
Leibniz Universität Hannover (LUH)
Forschungszentrum L3S
Appelstrasse 9a
30169 Hannover
Germany
Contact person: Stefan Dietze
E-mail address: [email protected]
The Open University
Walton Hall, MK7 6AA
Milton Keynes
United Kingdom
Contact person: Mathieu d'Aquin
E-mail address: [email protected]
Open Knowledge Foundation Limited LBG
Panton Street 37,
CB2 1HL Cambridge
United Kingdom
Contact person: Sander van der Waal
E-mail address: [email protected]
ELSEVIER BV
Radarweg 29,
1043NX AMSTERDAM
The Netherlands
Contact person: Michael Lauruhn
E-mail address: [email protected]
Open Universiteit Nederland
Valkenburgerweg 177,
6419 AT Heerlen
The Netherlands
Contact person: Hendrik Drachsler
E-mail address: [email protected]
EXACT Learning Solutions SPA
Viale Gramsci 19
50121 Firenze
Italy
Contact person: Elisabetta Parodi
E-mail address: [email protected]
Work package participants
The following partners have taken an active part in the work leading to the elaboration of this
document, even if they might not have directly contributed to the writing of this document or its
parts:
- LUH
- OU
- OKF
- Elsevier
- OUNL
- ELS
D1.2 – Challenge design
Page 3 of 26
Executive Summary
In this deliverable, we provide an overview of the LinkedUp challenge design. This includes the
structure of the overall competition and the rationale behind some of the decisions that were made in
the design process. There is a section that reviews the findings from our background research as well
as the core requirements and overall rationale for the challenge. There is also an overview of the
timeline and milestones for each of the three competitions. In addition, the data and infrastructure is
described including the repository and the processes that determined which datasets to include and
our plans to make them accessible to competitors. Finally, there is a section that describes the role of
the partner use cases that were selected for the competitions.
© Copyright lies with the respective authors and their institutions.
LinkedUp Support Action – 317620
Page 4 of 26
Table of Contents
1. Introduction ................................................................................................... 6
2. Background research and core requirements ......................................... 7
3. Challenge Overview...................................................................................... 9
3.1.
Introduction ............................................................................................................................. 9
3.2.
Overall rationale ...................................................................................................................... 9
3.3.
Competition tracks ................................................................................................................ 10
3.4.
Preliminary: LAK Data Challenge ........................................................................................ 11
3.5.
First competition: Veni.......................................................................................................... 12
3.5.1.
Rationale ........................................................................................................................ 12
3.5.2.
Entry requirements ......................................................................................................... 12
3.6.
Second competition: Vidi ...................................................................................................... 12
3.6.1.
Rationale ........................................................................................................................ 12
3.6.2.
Entry requirements ......................................................................................................... 13
3.7.
Third competition: Vici ......................................................................................................... 13
3.7.1.
Rationale ........................................................................................................................ 13
3.7.2.
Entry requirements ......................................................................................................... 13
4. Data, guidance and infrastructure ........................................................... 14
4.1.
LinkedUp Common Data Repository.................................................................................... 14
4.2.
Development support infrastructure ...................................................................................... 14
5. Use cases and focused tasks ................................................................... 15
5.1.
Introduction ........................................................................................................................... 15
5.2.
Veni use cases ....................................................................................................................... 16
5.2.1.
Teaching going social .................................................................................................... 16
5.2.2.
Educationizer ................................................................................................................. 16
5.2.3.
Simplify.......................................................................................................................... 17
5.2.4.
Supporting employees’ development via didactic guidance through learning materials
17
5.3.
Success criteria and risk mitigation ....................................................................................... 17
6. Timeline and milestones ............................................................................ 19
6.1.
First Competition: Veni (March 2013 - October 2013) ........................................................ 19
6.2.
Second Competition: Vidi (November 2013 – April 2014) .................................................. 21
6.3.
Third Competition: Vici (May 2014 – November 2014) ...................................................... 22
D1.2 – Challenge design
Page 5 of 26
7. Dissemination and promotion of the challenge ..................................... 23
7.1.
Target audiences ....................................................................................................................23
7.2.
Incentives to participate.........................................................................................................23
7.3.
Online promotion ..................................................................................................................23
7.3.1.
Websites .........................................................................................................................23
7.3.2.
Blogging & Social Media ..............................................................................................24
7.3.3.
Listservs .........................................................................................................................24
7.4.
Promotion at events ...............................................................................................................25
7.5.
First activities, responses and lessons learned.......................................................................25
8. Concluding remarks ................................................................................... 26
© Copyright lies with the respective authors and their institutions.
Page 6 of 26
LinkedUp Support Action – 317620
1. Introduction
In this deliverable, we will present the design of the LinkedUp challenge, including the overall
rationale for the challenge as a whole, as well as each of the three individual competitions.
Section 2 reviews the findings from our background research and the core requirements of the
challenge. This will include observations and lessons learned from previous related challenges. It
also contains a brief description of the core requirements for the individual competitions.
In Section 3, we explain the overall rationale for the challenge. In addition, we will explain the
details and components of each of the three competitions that comprise the challenge. For each of the
three competitions, Section 3 will give information on the rationale behind the competition, as well
as its entry requirements and evaluation criteria.
Section 4 is dedicated to the data and infrastructure of the challenge. This includes a summary of the
datasets that were included in the repository and the processes that went into determining which
datasets to include, how they were assessed and how they were classified. There is also information
on the support that LinkedUp will be offering to help entrants accessing and using the datasets.
These include the publishing of best practices and a “code clinic”.
Use cases and the focused tasks are the topic of Section 5. In addition to introducing each partner and
use case, the section will describe the evolving roles of the use cases through each of the three
competitions.
Section 6 contains the timeline and milestones for the challenge. This will include key dates around
the challenge and its individual competitions, as well as key internal milestones.
The dissemination plan will be introduced in Section 7. This section will include descriptions of
LinkedUp's target audiences and stakeholders and how we plan to address them at crucial parts of the
challenge. We will go into an overview of the outreach efforts such as the incentivisation to
participate, online promotions, and LinkedUp's participation at other related events.
We conclude the deliverable with a summary of the challenge design.
D1.2 – Challenge design
Page 7 of 26
2. Background research and core requirements
There is not such a thing as a best practice for designing a challenge on open educational data. Each
challenge or competition has its own context, peculiarities and requirements. Therefore, it is
important to identify the questions that need to be discussed, and to take informed decisions, based
on successful examples from the past.
Many competitions have been organized in the past, with various sizes and audiences. A first and
important observation is that for a competition on Linked Data the quality of the data is important1:
this does not only involve the quality and availability of the data, but also the coherence, relevance
and understandability of the data; as an example, a challenge at a workshop on Big Data 2 at the
UMAP 2012 conference failed to attract any submissions, because it was not clear what could be
done with the provided datasets.
In the LinkedUp challenge, we aim to address several relevant communities that are interested in
linked educational data, varying from students and researchers to startup companies and industry.
However, there are clear differences in how challenges are organized and presented in academia and
in industry.
Representative examples of an academic challenge are the workshop on “Data Mining on Linked
Data workshop with Linked Data Mining Challenge”3 and the Mashup Challenge at ESWC 20134.
Several characteristics can be observed:
 The challenge is embedded in a workshop or conference
 Participants need to submit a paper
 The challenges use academic terms as 'Call for Submissions', 'Program Committee',
'Notification of Acceptance'
By contrast, challenges that are initiated outside of the academic world use a very different
vocabulary. For example, the Thomson Reuters’ Innovation Challenge for Scientific and Scholarly
Research5 makes heavily use of graphics, and talks about ‘judges’ rather than a ‘program
committee’, about ‘solution seekers’ rather than ‘scenarios’ and ‘winner announcements’ rather than
‘notifications’. Non-academic competitions also put more emphasis on the prize money, the benefits
in terms of visibility and networking, and the ‘faces’ behind the program committee (or rather
‘judges’).
These observations have been intensively discussed during our project meetings and various email
exchanges. It was clear from the onset that we would not organize two separate competitions for the
academic community and for the industry. Therefore, it was essential to find a balance between both
approaches. As an example, the benefits listed in the recently launched announcement6 of the first
LinkedUp competition are a combination of ‘industry-oriented’ items (such as prizes and
networking) and ‘academy-oriented’ items (research questions and opportunities). Similarly, we
decided to complement the open track with use-cases and focused tasks that are explicitly endorsed
1
http://ukwebfocus.wordpress.com/2010/02/19/response-to-my-linked-data-challenge/
http://umbigdata.wordpress.com/
3
http://www.josemalvarez.es/web/2012/06/19/cfp-data-mining-on-linked-data-workshop-with-linked-data-miningchallenge/
4
http://2013.eswc-conferences.org/program/co-located-events
5
http://wokinfo.com/challenge/
6
http://linkedup-challenge.org/
2
© Copyright lies with the respective authors and their institutions.
Page 8 of 26
LinkedUp Support Action – 317620
by organizations such as the Commonwealth of Learning and Elsevier (guided by the WP5 work on
the use cases).
A particular issue was how participants were expected to provide their submissions. In the academic
world, it is common to write a paper, which positions their work in related work and shows the
scientific contribution of their system. In industry-oriented challenges, the focus is more on the
system itself and often a simple form-based submission is chosen. In the LinkedUp challenge, the
evaluation criteria (as developed in WP2) are very diverse and require some background information
on the submission, which has to be provided by the participants. Therefore, we developed a
submission template with predefined sections to be filled out and questions to be answered. The
template will be offered in two different ways: as an outline for an academic paper, and as a
submission form to be filled out.
Several decisions have been made and solutions found on issues, among others:





As discussed in more detail in Section 3 and 5, the challenge tasks need to be
sufficiently focused, but should not prevent academics and start-ups to send
submissions that are related to their 'core business'. It is important to show that the
focused tasks represent real needs of real institutions.
In Section 4 we discuss the data infrastructure (which is described in more detail in
Deliverables D3.1 and D3.1.2). It is important to have a data catalogue that contains a
variety of educational and non-educational resources. The data needs to be sufficiently
labelled and described, so that participants can make sense of it. These aspects have
been taken into account in the data assessment and topic alignment activities, which are
described in detail in D1.1.
Dissemination strategies are summarized in Section 7 and described in more detail in
the dissemination plan (D4.2). From various sources, we collected mailing lists, blogs,
social media and other platforms for promotion, and made a list of conferences and
other venues to address.
The incentives to participate were an important point of discussion in the context of
dissemination strategies. Three main incentives were identified: visibility and
recognition, networking opportunities, and pointers and support for R&D activities.
Apart from this, the cash prizes are a very visible and direct incentive.
The use cases, as described in Section 5, should provide the LinkedUp challenge
visibility and endorsement. The use-cases and focused tasks should be endorsed by ‘real
people’; the challenge committee should be represented by a small number of
prominent people
D1.2 – Challenge design
Page 9 of 26
3. Challenge Overview
3.1. Introduction
The LinkedUp challenge was started as a way to encourage creativity, innovation and
resourcefulness in finding ways to link and mash-up educational and cross-domain resources to
provide novel services for open and distance education. Specifically we hope to foster interest in the
use of Linked Data as a method that allows companies, universities, government agencies, and other
organizations to share their material and link it to other relevant material.
The challenge will be supported by LinkedUp partners that have been collecting and cataloguing data
related to education. The catalogue will be made available and accessible through search and
exploration services. More information about the data and support can be found in Section 4 of this
deliverable.
In total, there will be three competitions. The current schedule is displayed in Figure 1. The first one
launched on 11 March 2013. Participants can join in at each separate competition. Participants to
earlier competitions will be able to benefit from support and feedback from the LinkedUp team.



First competition: Veni (March 2013 - October 2013): Innovative scenarios, prototypes and
tools for analysing and integrating open educational Web data
Second competition: Vidi (November 2013 - April 2014): Challenging and innovative,
mature data-driven applications
Third competition: Vici (May 2014 - November 2014): Robust applications for large-scale
educational use-cases, offered and provided by LinkedUp
Fig. 1. The LinkedUp challenge competition timeline
3.2. Overall rationale
The LinkedUp partners decided to organize the challenge as three separate competitions for a
number of reasons. The competition structure is a hybrid, which we hope to offer the best of two
options: a full-scale single project alongside a smaller project with shorter duration. The primary
benefit of the structure is that it gains momentum and actually adds teams and submissions as it
progresses, as opposed to eliminating teams. The ongoing waves also provide more opportunity for
marketing and publicity. We also believe that the shorter engagement times for the competitions will
© Copyright lies with the respective authors and their institutions.
Page 10 of 26
LinkedUp Support Action – 317620
encourage more participation from private sector teams such as start-up companies and R&D teams
that would not likely be able to commit to the entire duration of the challenge.
In addition to the three competitions, the LinkedUp challenge will also be divided into two tracks: a
focused track and an open track (see Section 3.3).
The three competitions will build upon each other. For example, the use cases from Veni will help
shape the focused tasks in the Vidi competition. The Vidi submissions will become the large-scale
educational use cases in the Vici round.
The submitted projects will be assessed from a selected list of judges of the associated partners of the
LinkedUp project. The judges will be supported with an online scoring sheet to evaluate the projects
according to the same standards. The scoring sheet is an instance of the general Evaluation
Framework as developed in WP2 of the LinkedUp project (D2.1). The general Evaluation
Framework is a comprehensive collection of evaluation criteria, metrics and methods to assess the
participants of the data competition. Depending on a specific use case applied in the Veni, Vidi or
Vici competition, suitable criteria, metrics and methods will be selected from the Evaluation
Framework. For the Veni track a first version of the Evaluation Framework can be found in D2.2.1.
As the Evaluation Framework is one of the main outcomes of the FP7 LinkedUp project, it will be
further developed and amended according to the experiences collected after each data competition.
These upcoming content validation steps of the Evaluation Framework will change the framework to
more efficient and effective use. Each of the content validation reviews will be reported in an
amended version of D2.2.1 (D2.2.2, D2.2.3).
3.3. Competition tracks
Each competition will consist of two tracks: an open track, which is open to any kind of submission
that satisfies the entry requirements, and a focused track, in which participants aim to find solutions
to specific problems.
In the first competition, Veni, the tasks in the focused track are loosely defined as use cases, which
are given and endorsed by various institutions and people (the use cases are explained in more detail
in Section 5).
The use cases have been developed by the partners in cooperation with WP5. For each use case, the
proposer and the context of the task is given, along with a problem statement and suggested datasets,
followed by a list of example solutions that are sought.
In the Veni competition, we do not distinguish between 'open track' or 'focused track' submissions:
they will all be evaluated against the same criteria, as developed in WP2. However, participants who
choose for one of the use cases, may seek feedback from the institute or individual that endorses this
case. We expect that solutions for one of these problems may lead to future cooperation, either as a
more elaborate focused task in the Vidi competition, or as an activity outside of the scope of the
LinkedUp challenge. Furthermore, the use cases serve as illustrations of the kind of submissions that
LinkedUp is looking for, and serve as inspiration and context for open track participants.
Based upon the results of the Veni competition, the use cases will be elaborated into well-defined
focused tasks with expected results. These focused tasks will form the focused track of the Vidi
D1.2 – Challenge design
Page 11 of 26
competition. In the Vidi competition, both tracks will be independent from one another. The same
overall evaluation criteria will apply to both tracks, but focused track submissions will have as an
additional criterion to what extent the submission fulfills the given problem statement.
In a similar fashion, a selection of the Vidi results of the closed track will be chosen as starting points
for further elaboration in the final Vici competition. In the Vici competition, we expect robust
applications, which require time and effort to develop. For this reason, existing solutions will be
offered as a starting point, which can be elaborated by the developers themselves and – if the
developers agree – by other participants. In order to keep the final competition open for new
participants, it will still be possible to submit applications to an open track, which will have similar
entry requirements as the focused track. Both open track and focused track submissions will be
evaluated and treated equally, and there will be just one winning submission.
3.4. Preliminary: LAK Data Challenge
Prior to the Veni competition, a smaller, focused competition has been organized co-located with the
LAK 20137 conference on Learning Analytics and Knowledge, 8-12 April 2013 in Leuven, Belgium.
The goal of the competition, the LAK Data Challenge8, was to exploit the LAK Dataset, which
provides access to structured metadata from research publications in the field of learning analytics.
Challenge submissions should exploit the LAK Dataset for a meaningful purpose. This may include
submissions which cover one or more of the following, non-exclusive list of topics:
 Analysis & assessment of the emerging LAK community in terms of topics, people,
citations or connections with other fields
 Innovative applications to explore, navigate and visualise the dataset (and/or its
correlation with other datasets)
 Usage of the dataset as part of recommender systems
LinkedUp organized the LAK Data challenge in cooperation with the Society for Learning Analytics
Research. Organizing this challenge offers several advantages to the project and the LinkedUp
challenge:
 The opportunity to gain first hands-on experiences in running a competition on Linked
Data
 The possibility to promote the subsequent, larger LinkedUp Veni Competition among
the participants of the LAK Data challenge competition, and among the audience of the
challenge event, which is embedded in a LinkedUp tutorial on using Linked Data for
learning analytics9
 The submissions can be used as showcases and examples in the LinkedUp challenge
promotion material
 Solutions and approaches used by the LAK Data challenge participants can be used as
input for the best practice documentation in the LinkedUp WP3 support environment
The LAK Data challenge attracted 8 submissions, which received a light review. The extended
versions of the submissions, along with a paper that introduces the LAK Dataset, have been
7
http://lakconference2013.wordpress.com/
http://www.solaresearch.org/events/lak/lak-data-challenge/
9
http://linkedu.eu/event/lak2013-linkeddata-tutorial/?page_id=58
8
© Copyright lies with the respective authors and their institutions.
Page 12 of 26
LinkedUp Support Action – 317620
published online and open access at CEUR-WS10. More information about the outcomes and lessons
from LAK13 are included in D2.2.1 (Evaluation Framework).
3.5. First competition: Veni
The goal of the first LinkedUp competition is to gather innovative and robust tools that analyse
and/or integrate large scale, open Web data for educational purposes. We are expecting submissions
that demonstrate and promote the benefit of open Web data technologies for education to the widest
extent. Veni will be open for any end-user application that analyses and makes use of Linked Data or
Open Web Data for online learning.
3.5.1. Rationale
For each of the three competitions, there will be two tracks: a focused track and an open track. For
the Veni focused track, submissions will follow use cases and scenarios that LinkedUp has collected
from industry partners. The rationale behind offering a focused track is to be able to provide a
structure and ideas to guide teams that may need it. It is also intended to offer real world scenarios
that could be put into use. The open track was established to provide a platform for innovation and
creativity and the development of applications that may benefit scenarios unforeseen by the
challenge organizers.
3.5.2. Entry requirements





Creation of a demo, prototype or application that analyses and/or makes use of Linked
Data for online learning.
Use of data sets from the LinkedUp data repository and/or data from other sources.
Information about the application submitted via official challenge submission form.
A written abstract of about 200 words describing your application and its goals.
Demo made available to the review panel.
3.6. Second competition: Vidi
3.6.1. Rationale
For the second competition in the LinkedUp challenge, Vidi, entries should exhibit mature datadriven applications. It is important to note that entries can be extensions of submissions from the
Veni competition, or can be new entries. Participants carrying entries over from Veni will be able to
benefit from support and feedback from the LinkedUp team. The main difference between entries in
the competition and those from Veni, is that Vidi entries are expected to be moved significantly
beyond the prototype phase of an application. Designs and decisions for the Vidi competition may
evolve and change during the course of the prior competition.
Focused track entries should now be using data provided by the use case partners and solve specific
problems with solutions that meet stated expectations.
10
http://ceur-ws.org/Vol-974/
D1.2 – Challenge design
Page 13 of 26
3.6.2. Entry requirements




Creation of a data-driven application that analyses and/or makes use of Linked Data for
online learning.
Use of data sets from the LinkedUp data repository and/or data from other sources.
o Focused track entries using data from the specified use case partner.
Information about the application submitted via official challenge submission form.
A written abstract of about 200 words describing your application and its goals.
3.7. Third competition: Vici
3.7.1. Rationale
Vici will be the final competition in the LinkedUp challenge. As with Vidi, submissions can either be
extensions of entries in the previous competitions of new applications making their debut in Vici.
The submissions for Vici should be complete fully functional applications with extra emphasis on
their feature robustness and their scalability in handling Linked Data.
Focused track entries will continue using data provided by the use case partners and provide robust
solutions to the specific problems. As with the Vidi competition, designs and decisions for this
competition may evolve and change during the course of the prior competitions.
3.7.2. Entry requirements





Creation of a robust, large-scale data-driven application that analyses and/or makes use
of Linked Data for online learning.
Use of data sets from the LinkedUp data repository and/or data from other sources.
o Focused track entries using data from the specified use case partner.
Evaluation results that demonstrate the robustness and applicability of the application
in real-world contexts.
Information about the application submitted via official challenge submission form.
A written abstract of about 200 words describing the application and its goals.
© Copyright lies with the respective authors and their institutions.
Page 14 of 26
LinkedUp Support Action – 317620
4. Data, guidance and infrastructure
4.1. LinkedUp Common Data Repository
The first version of the LinkedUp Common Data repository, or Linked Education Cloud, is described
in deliverable D3.1.1. Available at http://data.linkededucation.org/linkedup/catalog/, this data
repository is a catalogue of datasets available as linked open data, provided both through a CKANbased registry of the datasets (see http://datahub.io/group/linked-education) and through a SPARQLbased data endpoint giving access to a fined grained description of the content of the datasets using
the VoID vocabulary.
The goal of the LinkedUp data repository is first and foremost to support the participants to the
LinkedUp challenge in obtaining data of relevance to their applications. It is however expected to
evolve into an essential resource for the community of developers and practitioners of open data in
education. The data in the repository will serve as suggested datasets, but other data is possible to
use. The data may not be explicitly about education but can be generic data or non-linked data.
This first version of the common data repository includes 22 data endpoint representing all together
146 distinct datasets explicitly about education (including academic programmes, educational
resources, research articles, educational institutions and facilities). These datasets have been
collected from existing repositories of linked open datasets.
In the future, the list of datasets will be extended with others not initially provided as linked open
data, and their descriptions enriched with higher level summaries of their content, educational
purpose and quality.
4.2. Development support infrastructure
The first building blocks of the development support infrastructure put in place in the LinkedUp
project are described in deliverable D3.2.1. The primary goal of this support infrastructure is to help
participants and prospective participants to the LinkedUp challenge in their development activities,
through taking benefit from the expertise and experience of the LinkedUp partners in exploiting the
relevant technologies in the education domain. One core component of this is the “code clinic”
environment, which will provide a channel and an opportunity for the challenge participants to share
specific development issues with the LinkedUp partners, and discuss their possible resolution. As we
want to make this facility as flexible as possible for potential participants, the code clinic is based on
two options: either giving LinkedUp partners access to existing code sharing facilities if they are
already in use, or rely on the project’s dedicated platform: a subversion- / bug tracking-based facility
hosted at the Open University. The support infrastructure is also expected to support the general
development of the area, beyond what is being done in the challenge. The second component of the
support infrastructure illustrates this, as its goal is to provide a global resource of bite-size tutorials
and use cases of the use web data technologies in education. This is achieved through the LinkedUp
DevTalk blog (see http://data.linkededucation.org/linkedup/devtalk/), with a dedicated editorial
workflow and content coming from the LinkedUp consortium as well as the experience we will
acquire through organising the LinkedUp challenge.
D1.2 – Challenge design
Page 15 of 26
5. Use cases and focused tasks
5.1. Introduction
To provide guidance and inspiration to challenge participants, we also provided sample use cases.
Developing and describing large scale scenarios and use cases for the large-scale exploitation of
open Web data, together with data mining and analytics techniques, is one of the objectives of the
project. We aim to exploit such use cases, expressing real needs from real, well-known institutions,
during the different competitions of the challenge as following:
 In the Veni stage, as inspiration and starting point
 In the Vidi stage, as a well-defined task with expected outcomes
 In the Vici stage, as elaboration of the Vidi winners/runners-up into a large-scale deployable
system
The initial collection for the Veni stage comprises four use cases expressed by the consortium and by
the associated partners. These use cases are meant to be an example and an incentive providing hints
about the kind of applications the challenge calls for. In some cases, also specific datasets are
provided. This is meant to make the challenge even more attractive as participants can access and
work with interesting and popular sources. The use cases help to drive applicants’ ideas and
developments towards concrete needs; the challenge targets useful and helpful solutions and not just
academic exercises. For each use case we present its proposer, context and challenges, suggest
datasets and possible solutions.
As far as the challenge progresses and competitions go, also the use and the intended outcomes
evolve. If for Veni the use cases were a kind of suggestion, in the Vidi competition they will be more
demanding. The use cases will present specific tasks and expected outcomes further specified.
At the final competition the applicants’ ideas should be deployed, tested and possibly adopted in
real-life large scale settings.
As for the other parts of the challenge, also the use cases collection will evolve and improve during
the project. Use cases for the different competitions don’t need to change every time. The most
interesting ones, both from the consortium and the applicants’ point of view, can be refined during
the different competitions. And new use cases can be added at each competition. Use cases are
sought via the consortium and the associated partners brainstorming, discussions, collaborations and
networking. The consortium also discussed about the possibility to call for use cases, in addition to
call for applications, but this hypothesis was discarded as it would have implied different modalities
for evaluations. In any case use cases are not binding the submissions: we remain always open to
applications unrelated to use cases.
In the following we shortly describe the four use cases presented for the Veni competition. For
further information, please refer to the challenge web site and to deliverable 5.1 (October 2013).
© Copyright lies with the respective authors and their institutions.
Page 16 of 26
LinkedUp Support Action – 317620
5.2. Veni use cases
5.2.1. Teaching going social
The use case was proposed by the Commonwealth of Learning11 (COL), an intergovernmental
organization created by the Commonwealth Heads of Government to encourage the development and
sharing of open learning/distance education knowledge, resources and technologies. COL is also a
LinkedUp associated partner.
The use case addresses educational challenges in developing countries, where there is not much
money to invest in education, but a strong need for high quality learning materials.
The need highlighted by this use case is the one to leverage social media in order to establish the true
relevance and value of a resource by analysing the personalized, enriched educational content of
social learners. This would allow a first screening of learning materials and would simplify teachers’
evaluation of resources. This simplified evaluation should be based mainly on three parameters:
location of information, educational value and social appreciation.
The data sets analyzed in this use case are comprised of learner inputs on OERs posted to YouTube
and Facebook in the form of ratings, likes and comments.
We invite applicants to submit ideas and prototypes addressing extraction and usage of information
from social media
5.2.2. Educationizer
This use case combines ideas, interests and needs expressed by: Jake Berger. Jake works at the BBC
Archive Development as Programme Manager for Digital Public Space. At the same time also the
project partner Elsevier expressed very similar interests. Elsevier is a world-leading provider of
scientific, technical and medical information products and services. The fact that similar needs and
interests were expressed by so different and relevant parties strengthens the relevance of the use case.
The starting point of the use case is the fact that most data and media is not explicitly ‘educational’,
but much of it has potential for use in education and research if surfaced in the relevant educational
context, and described using terms that are recognized and meaningful to the user.
The aim here is to support and facilitate the educational (re-)use of informative and multimedia
material not originally created for instructional purposes.
Interesting data sets under this perspective are clearly large informative databases and applications,
such as Wikipedia, Dbpedia, other linked open data sets, Youtube, Twitter hashtags, atlas, etc...
Respect to this use case, prototypes and applications for the challenge could address: supporting
mapping of non-educationally focused data to structured educational topics; vocabulary translation
from unfamiliar native vocabularies to those that are recognisable to educators, researchers and
learners; other and similar facilities that enhance the educational (re-)use of large heterogeneous sets
of data.
11
http://www.col.org
D1.2 – Challenge design
Page 17 of 26
5.2.3. Simplify
This use case combines ideas, interests and needs expressed by: Jake Berger (BBC Archive
Development).
Learners come in all flavours and levels of ability and experience, and everybody begins life
knowing nothing. People want to understand things but they are sometimes simply too new to the
subject to have learned lots of domain-specific words, or even simply too young to have learned
enough words at all. Learning should try to be as accessible and attractive to the novice as it is to the
expert.
Data sets that can be used for this use case are any kind of sector- and domain- specific datasets, such
as databases of scientific research publications.
This use case suggests “simplifier” applications that for example analyze and summarize complex
abstracts and research in simpler terms, make research papers easily readable by less-advanced
learners and easier to auto-translate, and similar facilities.
5.2.4. Supporting employees’ development via didactic guidance through
learning materials
This use case originates from consortium discussions and brainstorming. It appears to be close to
actual needs of SMEs and SMEs employees. It also describes themes and issues that are frequently
encountered in discussions about MOOCs.
At a company, employees may wish or may be invited by their company itself, to attend online
courses to improve their knowledge, keep up-to-date and have the opportunity to progress in their
career.
What would be helpful here is a guidance system, i.e. to suggest which course would make sense to
take next or to work on simultaneously; and/or basing on own learning/career objectives, previous
learning (prerequisites), etc. Another valuable support would be that of remediation to relearn and
reflection. Another useful support would be a kind of formal recording / history of attended MOOC,
providing some sort of progression timeline, or updated resume.
Many institutions and organizations provide MOOCs, such as Coursera, edX, Udacity, etc... that can
be used as datasets for this use case.
Applicants submissions could propose support and guidance across MOOCs and OERs, i.e. to
suggest which course would make sense to take next, or to work on simultaneously; more crowdbased search and structuring facilities of existing offerings (journals, conferences, books, etc…) and
new online tools.
5.3. Success criteria and risk mitigation
Use cases are elaborated on the basis of consortium experience, knowledge and reflection about TEL
emerging needs, current trends and discussion topics with relevant stakeholders. This leads to use
cases that correspond to real life contexts and actual needs and challenges that institutions and people
are facing.
© Copyright lies with the respective authors and their institutions.
Page 18 of 26
LinkedUp Support Action – 317620
Providing several use cases mitigates the risk related to single use cases not encountering much
interest from participants. Topics, description means and requests of the use cases will be also remodulated, competition by competition, on the basis of the feedback received by participants, both
via their submissions and via the support environment.
D1.2 – Challenge design
Page 19 of 26
6. Timeline and milestones
The global timeline of the three competitions is dictated by the Description of Work. The first six
months of the LinkedUp project were planned as preparation time for the overall challenge design
and the first competition. Following the preparation time, the three competitions will be launched
with an interval of six months.
Apart from the launch dates, the submission dates and the final events, the timeline needs to ensure
that between launch and submission there will be sufficient activity to maintain the attention of the
community. Therefore, each competition will have a gradual disclosure of the data, documentation
and submission instructions, which will then be announced through the various communication
channels.
In order to increase visibility and attention, supporting events have and will be planned at relevant
key conferences and fairs. As the exact planning of the dates and locations of these conferences and
events is usually not known until one year in advance, the planning for the Vidi and Vici competition
is tentative and needs to be revised in the preparation phase for these competitions. For the Veni
competition, we selected various venues that will be used for promotion of LinkedUp and the
LinkedUp challenges.
6.1. First Competition: Veni (March 2013 - October 2013)
The launch of the first competition was scheduled to be early April (month 6 of the project lifetime).
However, when looking for suitable venues for the submission presentations and award ceremony, it
became clear that we had to target an event early September, as there would be no suitable events
earlier than December 2013.
In order to have sufficient time for reviewing the submissions and to send out notifications well
before the event takes place (to allow participants to arrange their travels in time), a time span of at
least two months is required between submission deadline and the closing event. Due to the
European Summer Holidays, which start early July in most countries and regions, the submission
deadline was set to be the end of June 2013. Subsequently, in order to have sufficient time between
launch and submission, we decided to launch the Veni challenge two weeks earlier than originally
planned – mid March.
Key dates





18 March 2013: Launch of the challenge
13 May 2013: Release of the comprehensive LinkedUp dataset
27 June 2013: Submission deadline
16 August 2013: Notifications and Nominations
17 September 2013: Presentations and award ceremony at the OKConf in Geneva,
Switzerland. (http://okcon.org)
© Copyright lies with the respective authors and their institutions.
Page 20 of 26
LinkedUp Support Action – 317620
Related conferences and events
On the internal LinkedUp Wiki we have a long list of conferences that are directly or indirectly
focused on Linked Data, learning analytics, semantic web technologies or technology-enhanced
learning. The list is regularly updated.
From the list, we selected four venues that we will specifically target. These venues are listed below.
LAK 2013 - Learning Analytics and Knowledge. 8-12 April 2013, Leuven, Belgium.
At this conference, a tutorial was given on the use of linked educational data. Related to the tutorial,
we created a 'LAK Data Challenge' (see Section 3 for more details), which attracted 8 submissions.
Both the tutorial and the Data Challenge aim to show the LAK community that the LinkedUp
competition is something that they really should want to join.
The Next Web Conference, Europe 2013. 25-26 April 2013, Amsterdam, The Netherlands.
We sponsor and participate in this event to effectively target the business sector, specifically startups
and SMEs.
WWW 2013 13-17 May, Rio de Janeiro, Brazil. As organizers of LILE 2013, the 3rd international
workshop on Learning and Education with the Web of Data, we will promote LinkedUp at this big
conference.
ESWC 2013 - Extended Semantic Web Conference. 26-30 May, Montpellier, France. We sponsor
this conference in exchange for promotion onsite and on the website.
In addition, we plan to promote LinkedUp at the JTEL Summer School (27-31 May, Cyprus), in
order to specifically target doctoral students.
Internal deadlines and milestones
April 2013 (M6 of the project) is the deadline for several deliverables that focus on a specific aspect
of the LinkedUp challenge. Consequently, April 2013 is an important milestone for the Veni
Competition.







State-of-the art and data assessment (D1.1)
Challenge design (this deliverable)
Evaluation criteria and methods, evaluation framework (D2.1, D2.2.1)
Common data repository (first version) (D3.1.1)
LinkedUp support environment (D3.2.1)
Dissemination plan (D4.2)
Press releases (D4.3)
In addition, there are three updates to the LinkedUp dataset planned:
 End March 2013: Expanded Linked Education catalogue, launch of the LinkedUp
DevTalk blog/forum
 End April 2013: Release of the Linked Education tagset for cataloguing educationrelated datasets
D1.2 – Challenge design

Page 21 of 26
End May 2013: Release of the advanced catalogue, including search and navigation
facilities
Following the submission deadline of 27 June 2013, we plan the following review procedure:
 4 July 2013: review assignments sent to the program committee
 9 August 2013: deadline for reviews
 16 August 2013: final decisions on acceptance and ranking of the submissions;
notifications sent out.
6.2. Second Competition: Vidi (November 2013 – April 2014)
Key dates
The preparations for the Vidi competition will start in September 2013. Main focus of the
preparation activities will be the creation of the focused tasks, building upon our experiences and the
results of the Veni competition. A second preparation task will be to determine the entry
requirements for this round. As discussed in Section 3, the Vidi round will request more mature
submissions than the Veni round. The exact requirements should balance between our goals and
ambitions, as described in the DoW and in section 3, and realistic expectations, based upon our
experience with the Veni challenge.
The exact dates for the Vidi competition need to be determined during the two preparation months as
well. Analog to the Veni challenge, the following dates will be used as a guideline.





November 2013: Launch of the challenge
December 2013: Release of the focused tasks and the corresponding datasets
February 2014: Submission deadline
March 2014: Notifications and Nominations
April or May 2014: Presentations and award ceremony
Related conferences and events
The award ceremony will be co-located with an existing conference or other event. At the moment of
writing, details of conferences in the period April-May 2014 are not yet known. The most probable
candidates are the Extended Semantic Web Conference, ESWC 2014, and the Learning Analytics
and Knowledge Conference, LAK 2014.
Based on our list of events, the following events should at least be targeted for promotion of the Vidi
challenge and presentation of the Veni results:
 ISWC 2013 - International Semantic Web Conference (21-25 October 2013, Sydney)
 Online Educa (4-6 December 2013, Berlin)
Internal deadlines and milestones
These have to be specified, once the exact dates and deadlines have been fixed.


October 2013: creation of the focused tasks, specification of entry requirements,
selection of a venue for the award ceremony.
December 2013: release of the use cases and associated data
© Copyright lies with the respective authors and their institutions.
Page 22 of 26

LinkedUp Support Action – 317620
March-April 2014: submission review and notification
6.3. Third Competition: Vici (May 2014 – November 2014)
Key dates
Similar to the previous two competitions, the Vici challenge will have a lifetime of 6 months. At the
moment of writing it is too early to fix specific dates, as we do not have the dates for conferences
and other venues at the end of 2014. The global timeline for the Vici challenge will be:




May 2014: Launch of the challenge
End September 2014: Submission deadline
End October 2014: Notifications and Nominations
November 2014: Presentations and award ceremony
D1.2 – Challenge design
Page 23 of 26
7. Dissemination and promotion of the challenge
7.1. Target audiences
There is a clear split between two target audiences that will be addressed with the challenge. On the
one hand, we are aiming to get many submissions from people in academia. On the other hand, we
aim to engage businesses and provide incentives for them to participate. In the dissemination plan,
we more elaborately define how we target these different audiences.
These are the different stakeholder groups that are most important in targeting for the challenge:
Academia
 PhD Students (CompSci, library science)
 Providers of OER content (participating
and dissemination)
 Researchers in information science
 Computer Science researchers
 Technology Enhanced Learning (TEL)
researchers (CompSci)
Industry
 Startup companies
 Entrepreneurs
 Small and medium sized software
companies
Furthermore, we work closely with media outlets, as a channel to address general public wider
dissemination.
7.2. Incentives to participate
When considering the incentives for people to participate in the challenge, there are varying
incentives which we deem more or less important for the challenge. Here, we are listing them in
order of importance:
Academia
 Academic reward (eg. published papers)
 Recognition from peers
 Visibility among their research
community
 Visibility among leaders in their field
 Prize money
Industry
 Visibility among the linked and open data
community
 Prize money
 Networking opportunities across industry
and with possible investors
7.3. Online promotion
7.3.1. Websites
The LinkedUp consortium has two primary public-facing websites. One dedicated to the LinkedUp
project as a whole12 and one dedicated to the LinkedUp challenge13. The site holds detailed
information about the Work Packages and their roles. There is also a page for each of the following
groups: consortium partners, associated partners, and the advisory board. Pages provide lists of the
12
13
http://linkedup-project.eu/
http://linkedup-challenge.org/
© Copyright lies with the respective authors and their institutions.
Page 24 of 26
LinkedUp Support Action – 317620
organizations and a brief statement about their role in the project. In an effort to meet some of the
educational objectives, the site also contains a bibliography of publications related to Linked Data
written by the consortium partners. The ‘Contact’ page contains links to our various communication
channels such as Twitter and the mailing lists.
The official dissemination vehicle for information about the challenge will be the LinkedUp
challenge website. This site includes all the information that prospective entrants would need to
participate in the challenge. The site is divided into nine sections:
What is it?
Join now!
How to participate?
Why participate?
Data and support
Review Process
Get in touch!
Background
Use cases
7.3.2. Blogging & Social Media
In addition to the websites, the LinkedUp consortium also has a blog. Content will include news and
updates about both the LinkedUp challenge and other LinkedUp activities, partner attendances at
events/conferences, as well as other related topics. The dissemination team has made it a priority to
keep the LinkedUp blog updated regularly, and a schedule has been created for project partners to
contribute posts. Each partner will be expected to post circa every 6 weeks, although more frequent
posts from all partners would be desirable. In addition to the scheduled partner blog posts, the
community coordinator will also solicit for guest blog posts from associate partners and other
interested parties.
Blog posts will be publicised over social media channels such as Twitter, through the LinkedUp
mailing lists and through the RSS feed for the blog. We will also ask our project partners and
associate partners to advertise these posts through their relevant networks and dissemination
channels. We will collect a central list of these dissemination channels to ensure we have an
understanding of the breadth of these channels, and prevent too much duplication of effort.
7.3.3. Listservs
LinkedUp has established three LinkedUp project listservs to ensure effective communication
between various stakeholders. One designed exclusively for internal communication amongst the
consortium partners, another private mailing list for the LinkedUp network of consortium partners,
associate partners and advisory board, and finally a public mailing list.
D1.2 – Challenge design
Page 25 of 26
When blog posts are published on the LinkedUp project website, members of the linkedup-network
mailing list will be notified by the community coordinator, and will be asked to forward this through
their networks and dissemination channels. This will drive traffic back to the LinkedUp website and
raise awareness of the project, challenge and three competitions.
The Dissemination team has also collected external email listservs that are likely to contain target
audiences. These listservs will serve as another channel for the community coordinator to announce
upcoming LinkedUp events at conferences and significant milestones related to the challenge. This
list shall be dynamic, and will likely expand over the course of the LinkedUp project, and will be
maintained in a living document. Consortium partners will be expected to add further possible
listserv targets throughout the project.
7.4. Promotion at events
To support goals and outcomes of the initiative and draw attention to the challenge, the LinkedUp
project has put forward a strategy for involvement and participation at conferences and events related
to open data for education, Linked Data and the semantic web. The types of activities that we are
currently planning include: hackdays, workshops, tutorials, panels, and keynotes and presentations.
We are also making a concerted effort to be diverse in the types of events that we participate in. In
terms of event sponsoring, we strive for a distribution between academic/science themed events and
industry events. In the current planning for 2013 event participation, the focus of LinkedUp is on
participating with existing events, rather than organizing new ones. The rationale behind this being
that the other events already have momentum and are attracting audiences.
7.5. First activities, responses and lessons learned
A separate site (www.linkedup-challenge.org) has been set up to promote the challenge, with a
design to try and put the challenge, and in particular the Veni competition across, as an attractive
competition for both research and business communities. Initial feedback suggests that in order to get
people enthusiastic about the challenge, the content on the site needs to be shortened further and
made more to-the-point. A dedicated community coordinator, as part of the dissemination work
package WP4, is at the time of writing working on implementing those changes in collaboration with
the project team.
The LAK Data challenge in April has already attracted 8 submissions and has provided materials that
will be used as examples for the dissemination of the Veni competition. In particular a short video
that was created by the ‘Cite4me’ submission has received positive feedback and will be used in
other dissemination activities.
© Copyright lies with the respective authors and their institutions.
Page 26 of 26
LinkedUp Support Action – 317620
8. Concluding remarks
The LinkedUp consortium is sponsoring the LinkedUp challenge as a means to inspire innovative
entries of applications that make use of open data in order to provide novel services for open and
distance education. In this report, we gave an overview of the planning and rational that we are using
to guide the design of the challenge.
The challenge is divided into three competitions which will take place one after the other. The
rationale behind creating the three competitions is that it provides different levels of entry for teams
based upon their strengths and limitations including technical prowess and the ability to commit a
significant amount of time. In addition, having the three phases not dependent on one another means
that the number of entries can grow through the duration of the challenge, rather than get smaller
with elimination rounds.
To foster the overall goal of the LinkedUp project of educating and sharing knowledge about using
Linked Data and open web data, the LinkedUp challenge will be providing guidance and tutorials to
assist entrants.
In an attempt to make the entry process more meaningful in expressing real world scenarios, the
challenge will make use of use cases provided by the Commonwealth of Learning, Elsevier, and the
BBC. These use cases will evolve throughout the duration of the competitions and serve as
foundation for focused tasks in which entrants will be given more guidance for their applications.
The open track is for entries that are seeking innovations solutions and applications for different
problems and use cases that competitors bring forward.
The dissemination plan for the LinkedUp consortium is designed to inform and engage the target
audience of the LinkedUp initiatives. This includes keeping the audience abreast of information
related to the LinkedUp challenge as well as other opportunities to participate with LinkedUp
members such as attending tutorials and other events at conferences. One of the core activities that
went into the dissemination plan was the identification and description of our target audiences. We
hope to address them through a largely online campaign that includes a regularly updated website,
regular blogs and social media updates, press releases, and in-person engagements.