CLARIN tools for
workflows
Overview
Objective of this document


Determine which are the responsibilities of
the different components of CLARIN
workflows.
Explain what is expected from the CLARIN
Workflow tools. Requirements.
Firstly, what is a workflow?




Workflow is a definition of a process in a certain
language. It is done in a non-ambiguous way.
A workflow is only the definition of a process.
Since the workflow definition is made in a nonambiguous way, any software can execute it
expecting the same results. It means that workflows
can be shared between different applications.
Workflow may be a XML file that a user can write
without any other help.
Workflow tools

Since a workflow is only a process
description, we need 2 other components.
 Workflow Engine: Application that process
(executes) workflow descriptions.

Workflow Editors: Application for creation of
workflow descriptions.

Both applications can be in the same piece of
software. Workflow Workbench.
Workflow Engines: Expectations


Engines are applications that execute
workflows in the way they are described.
Many engines can be developed. Some with
GUI for desktop computers some other hosted
in servers with schedule features, etc...
Workflow Editor: Expectations





For advanced users of CLARIN, a text editor could be enough for a
workflow creation.
But CLARIN objective is easiness. In this way to create a workflow in
CLARIN must be really easy.
The editors in CLARIN must help as much as possible to the user. This is
done, for instance, querying the registry and taking decisions behind the
scenes.
Example: User connects two web services using first’s output with
second’s input. Metadata stored in the registry specifies that this operation
is not possible without a conversion. Also the registry contains info about
converters available. The editor must be aware of that and introduce a
conversion tool between the web services. This can be done without user’s
interaction at all.
SMART editors are possible since the Registry will contain all the
required metadata for taking decisions automatically.
Example

User wants to connect 2 web services.





Web service X
Web service Y
X’s output needs to be Y’s input.
But X’s output is not in the proper format for
being Y’s input.
The workflow editor must be able to find a
converters.
Example. Finding a converter

The editor needs to do the following tasks:




Get information about web service X. Which
format has X’s output?
Get information about web service Y. Which
format requires Y’s input?
Find a converter. How can data in format X be
transformed in format Y?
All this information must be stored and
accessible in the registry
Example. Get information of X

It is required to query the registry. Still it is
not decided how this query will be but
intuitively it should look like:

specObject = GetWebServiceSpecification(X_persistent_id)

This call will return the specification where
explains that X’s output format follows the
standard A.
Example. Get information of Y

Like previously with X, the editor queries the
Registry for Y’s specification.

specObject = GetWebServiceSpecification(Y_persistent_id)

This call will return the specification where
explains that Y’s input format follows the
standard B.
Example. Finding a converter


Now the editor needs to find a converter from
standard A to standard B.
It will be done using the Registry as well. Also it is
not developed yet but it will be something similar to:

ResourceList = GetResourceList(“/standards/converters”)

The editor has downloaded the list of converters
(resources under the standards/converters branch). Now
inspecting this list of resource descriptions, the editor will
find the required converter (if any).
Example. Introduce the converter in
the workflow

Before the insertion of the converter in the
workflow, depending on the user’s profile:





the user will be asked for confirmation or
the user will be noticed of different converters if more
than one is available or
simply won’t be notified at all (novice users)
The editor introduces the converter in the workflow
between X and Y.
The workflow description will keep this converter
information. Engines executing this workflow will
need to know about this conversion.
Workflow editor: Requirements





Graphical representation of all workflow description
elements. (loops, if then clauses, parallelization,
exceptions, etc…)
SMART editor.
Different user profiles (Advanced users will want to
know more about automatic decisions taken by the
workbench while novice users won’t want to decide
anything)
Drag & Drop features.
Registry browsing
Workflow workbench.


The editor can help even more to the user if it
is aware of the workflow execution results.
That’s why it is recommended to develop a
Workbench (Workflow Editor + Workflow
Engine together).
Also for basic users it is easier when
everything is centralized in just one
application.
Workflow Workbench: Requirements




All Editor requirements
Debugging tools: breakpoints, Start-pause
execution and inspection of intermediate
steps.
Partial workflow execution and reuse of data
obtained in previous executions. (To avoid
repeating calls to web services while testing)
Output console
Very important. SMART applications



Different profiles in CLARIN applications. Don’t
bother novice users with advanced questions.
Applications remember similar situations in the past.
“You connected two web services similar to this one
using converter AtoB. Do you want to do it again?”
Use of community intelligence stored in the registry.
“250 users connected these 2 web services in other
projects using converter AtoB, do you want to do it
as well?”