Project: Title: Ref: Date: Version: Author: State: Circulation: Pages: ErcatoLT server API LivisTR-ELT01 June 28, 2001 1.0r Dr. Falk Langhammer Living Pages Research GmbH [email protected] Released, confidential © 2001 Living Pages Research GmbH restricted 18 Table of contents 1. Wire protocol....................................................................................................................................3 1.1 Wire protocol testbed.................................................................................................................4 2. Constructor API................................................................................................................................4 2.1 Terms.........................................................................................................................................4 2.2 Manage users.............................................................................................................................4 2.3 Open Job....................................................................................................................................6 2.4 Submit application.....................................................................................................................7 2.5 Submit job..................................................................................................................................8 2.6 Commit Job................................................................................................................................8 2.7 Rollback Job..............................................................................................................................9 2.8 Retrieve stati of Jobs..................................................................................................................9 2.9 Retrieve result of Job...............................................................................................................12 2.10 Close job................................................................................................................................12 2.11 Accounting.............................................................................................................................13 2.12 Delete job...............................................................................................................................13 3. Destructor API................................................................................................................................14 4. Mediator API..................................................................................................................................14 4.1 Manage users...........................................................................................................................14 4.2 Listing jobs...............................................................................................................................14 4.3 Changing job states..................................................................................................................15 4.4 Deleting jobs............................................................................................................................15 4.5 Uploading public client resources............................................................................................16 4.6 Updating the ercato client software release.............................................................................16 4.7 Updating the ercato client look&feel.......................................................................................17 –2– This document describes the Application Programming Interface (API) of the “ErcatoLT (Light) server”. This API is incompatible to the specifications in LivisTR-ERC01 and LivisTR-ERC02. It replaces document LivisTR-EOS01 for the ErcatoLT server. The API how to use a specific client application for capturing of data using this electronic outsourcing API is described in document LivisTR-FB01. 1. Wire protocol The wire protocol defines how to communicate with an ErcatoLT server. The following example clarifies the basics. Refer to documents ftp://ftp.isi.edu/in-notes/rfc2616.txt and ftp://ftp.isi.edu/in-notes/rfc2617.txt for details. All examples will appear in a dedicated typesetting. Only bold-blue text is variable and specific to an example, the black text is as is and part of the specification. A Request is sent to the ErcatoLT-Server at some URL, e.g.: http://ejb.living-pages.de/ercatolt/command.jsp like the following: Note: In all following examples, we use the URL fragments ejb.living-pages.de and ejb.living-pages.de/ercatolt/ as examples only. Replace in all instances by the actual correct value! POST /ercatolt/command.jsp HTTP/1.1 Host: ejb.living-pages.de Content-Type: application/x-www-form-urlencoded; charset="iso-8859-1" Connection: Keep-Alive User-Agent: Mycompany/ErcatoLTlayer Content-Length: 4711 ...further HTTP header fields... Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== command=OpenJob&jobName=test1&details=nur+ein+%3Cb%3ETest%3C%2Fb%3E%21 This request may also contain a valid session id stored in an HTTP cookie as obtained after a login operation. The example printed above submits the following parameters: Relative URL: Method: Parameter: command jobName details command.jsp POST Value (before url-ecoding): OpenJob test1 nur ein <b>Test</b> Comment: Values must be url-encoded which escapes characters such as “ “ or “<”. Parameter/Value-pairs are submitted as “url-encoded-Parameter=url-encoded-Value” and separated by the “&”-character. They form the body of a POST-request or are appended after a “?”-character to the URL of a GET-request. Note that GET-requests have limited length. The authorization is according to the HTTP-basic athentication specification. This means that the “Authorization” header contains, after the string “Basic “, the string “username:password” with actual username and password inserted before and after the colon-”:” character, respectively, and –3– encoded using the base64-encoding method (which is also used to encode MIME mail attachments). With missing authorization the server will reply with a “401 authorization required” error message. The Http-Response (using the same socket connection) may read as follows: HTTP/1.1 200 OK Content-Type: text/xml; charset="iso-8859-1" Content-Length: 134 <?xml version="1.0" encoding="ISO-8859-1" ?> <response request-url="http://ejb.living-pages.de/ercatolt/command.jsp"> <job-name>test1</job-name> <job-key>4822</job-key> <state>ok: creation started</state> </response> A server failure will cause an Http-error code (outside 200 and 299) or a response with a state not starting with “<state>ok”. 1.1 Wire protocol testbed In order to facilitate the development of constructor software, the ErcatoLT server implements a testbed to help clarifying protocol details. This testbed is a straight web browser access. Assuming the server's URLs read like http://ejb.living-pages.de/ercatolt/command.jsp, then the testbed is used as follows: v v v v v Open Microsoft Internet Explorer 5.0 or 5.5 (it must be a browser with XML support implementing the XSL-Draft (pre-final) specification!) Enter URL http://ejb.living-pages.de/ercatolt. Login as a user with a constructor rôle, typically one such pre-configured user is called 'elboss'. Click onto 'list' and 'new job' menue items. Play around with the options available to constructors. The following sections will only describe request-/response pairs transmitted over the wire protocol. 2. Constructor API The ErcatoLT server differentiates between constructor, destructor and mediator. The construcor creates new items (jobs), the destructor consumes those items (processes jobs), and the mediator is in charge of supervising the flawless operation (eg., manually changing job states). 2.1 Terms The term constructor refers to the role of users constructing new jobs.. 2.2 Manage users Users are managed in a single file, referred to as “principals.xml”. The role of a user is defined using a tag, <group-membership group="rôle" /> whererôle is one out of: –4– rôle Meaning: Administrator User with permission to manually alter jobs. Should be distinct from Constructor. Constructor User with permission to create jobs and retrieve results. There may be more than one Constructor. Destructor User with permission to work on jobs. There are typically many such users. File “principals.xml” has the following structure: <?xml version="1.0"?> <!DOCTYPE principals PUBLIC "//Evermind - Orion Principals//" "http://www.orionserver.com/dtds/principals.dtd"> <principals> <groups> <group name="Destructor"> <permission name="rmi:login" /> <permission name="com.evermind.server.rmi.RMIPermission" /> </group> <group name="Constructor"> <permission name="rmi:login" /> <permission name="com.evermind.server.rmi.RMIPermission" /> </group> <group name="Administrator"> <permission name="rmi:login" /> <permission name="com.evermind.server.rmi.RMIPermission" /> </group> </groups> <users> <user username="eladmin" password="ela"> <group-membership group="Administrator" /> </user> <user username="elboss" password="elb"> <group-membership group="Constructor" /> </user> <user username="elworker" password="elw"> <group-membership group="Destructor" /> </user> <user username="hans" password="wurst" realname="Prof. Hans Klein"> <description>Our man for troubleshooting</description>> <group-membership group="Destructor" /> </user> <!-- ...more workers (Destructors) go here... --> </users> </principals> To change or add users, file “principals.xml” is modified locally and stored under the following URL: Principals-URL: Paßwort: ftp://[email protected]/principals.xml <mediator-password-A> (not disclosed) Time when changes to users becomes effective: immediately after upload! This URL must be read as follows (some browsers understand URLs as this one directly, though): v Open FTP client –5– v v v v v v Connect to ejb.living-pages.de Login as user elusers Provide password as is not disclosed here Set FTP-binary mode Put or get file principals.xml Quit FTP client This means that users for the ErcatoLT server are managed without using the wire protocol as described above. We will use URLs starting with ftp: again in this document and their meaning is according to the description here. 2.3 Open Job Creation of a job involves four steps: 1. 2. 3. 4. Open job Submit application Submit job Commit job First, a job must be opened for further manipulation: Relative URL: Method: Parameter: command stylesheet command.jsp POST Status: Value: OpenJob fixed jobName userGroup required volume deadlineMonth deadlineDay deadlineYear deadlineHour deadlineMinute details Comment: will be inserted (if present in request) into XMLreply for browser-friendly manipulation of response a unique, non-empty name only characters a-z, A-Z, 0-9, “.”, “_” are allowed names of users, separated by a if empty or missing, all users will see the job. “|”-character. Otherwise, only users listed in userGroup will see it. a hint about the size of a job a time value such as “60 min” is preferred. 1-12 deadline is currently ignored and serves informational purposes only. It may become a required parameter in a future release! 1-31 2000-2475 0-23 0-59 content of an HTML-page if present and non-empty, a “details”-link to this page is inserted into the job-listing. The details' content may be arbitrarily complex HTML source, including further links (<a href=””>-tags). Parameters are submitted such as described in 1. Wire protocol on page 3. In case of success, the server replies: –6– Type: Root-tag: Sub-tag: job-name job-key state XML response Status: Value: Comment: same as jobName essential a unique key store for all further reference to jobKey! ok: creation started Any value not starting with “ok” is an error condition! 2.4 Submit application Now we need to prepare the client's working environment. This basically consists of a number of files. As specified in document LivisTR-FB01, the following files are required or may be supplied: v v application.xml client.xsl Relative URL: Method: Parameter: command stylesheet jobKey applicationClient clientStylesheet command.jsp POST Status: Value: SubmitApplication fixed required the job's key content of file application.xml content of file client.xsl Comment: this was returned from previous request. this must be a valid XML-document and properly url-encoded to preserve the content. this must be a valid XML-document and properly url-encoded to preserve the content. The client stylesheet is immediately applied to application.xml using a stylesheet processor. Any error occurring during stylesheet processing will cause an unsuccessful server response with stylesheet processor error messages embedded into the state-tag of the response. Subsequent “SubmitApplication”-requests may be issued until a successful response is eventually obtained. In case of success, the server replies: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... You may find a stylesheet processor's error message appended to “failed: “ in the state. –7– 2.5 Submit job As specified in document LivisTR-FB01, the following files are required to submit a job: v v paperforms.xml job.xml Relative URL: Method: Parameter: command stylesheet jobKey paperForms job command.jsp POST Status: Value: SubmitWork fixed Comment: required the job's key content of file paperforms.xml content of file job.xml this must be a valid XML-document and properly url-encoded to preserve the content. this must be a valid XML-document and properly url-encoded to preserve the content. This file contains the data which shall be captured and this is the file which is retrieved after the job is done. The image URLs within job.xml cannot be local on an ErcatoLT server if there is no server operation to upload images onto the server. This document describes no such operation. In case of success, the server replies: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... 2.6 Commit Job We now commit our job: Relative URL: Method: Parameter: command stylesheet jobKey command.jsp POST Status: Value: CommitOpenJob fixed required the job's key In case of success, the server replies: –8– Comment: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... It is at this moment that the job becomes available for processing. Until after a job is committed it cannot be processed by workers. 2.7 Rollback Job We also can roll our job back: Relative URL: Method: Parameter: command stylesheet jobKey reason command.jsp POST Status: Value: RollbackOpenJob fixed required the job's key a text specifying why the job was rolled back Comment: bug: may be required (to be tested) The purpose of this request is to alter job.xml before it gets eventually committed. A forthcoming version of the ErcatoLT server may allow arbitrary modifications before the commit operation. Bug: Only the last step (submit job) can be rolled back. In case of success, the server replies: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... 2.8 Retrieve stati of Jobs After a job was committed it will be very likely that a constructor wishes to monitor progress. This is possible by issueing the following request: –9– Relative URL: Method: Parameter: jobKey age nrecs firstrec filter jobs.jsp ⇐ different from all other requests! POST or GET Status: Value: Comment: required the job's key only required for 'filter=byJobKey' the jobs' maximum age in Each time a job gets modified, its age is set to zero. minutes (integer). The age of a job with a deadline which has not yet passed is kept zero. Specifying age results in a shorter and faster resonse ignoring old jobs. Missing age (or age=0) disables this feature. The number of records to be returned at most. This the maximum number of may reduce transmission time but not necessarily jobs to be returned response time. (integer). Missing nrecs (or nrecs=0) disables this feature. position of first record to be To be used in conjuction with nrecs. This allows to returned (starting at 1) retrieve subsequent records. Missing firstrec (or firstrec<1) disables this feature. inCreationBy required a constructor's jobs in creation (not committed) pendingFor a constructor's committed jobs with no worker inProgressFor a constructor's jobs with a processing worker finishedFor a constructor's jobs with completed results unacceptedBy a constructor's jobs with retrieved results archivedFor a constructor's closed jobs byConstructor all constructor's jobs byJobKey a single constructor's job with a given job-key Note that it is not possible to retrieve stati of other constructors' jobs. In case of success, the server replies with the stati of zero, one, or multiple jobs in the following way: – 10 – Type: XML Root-tag: jobs Sub-tag: Attribute: title user list-options selection selection/found fields fields/field Value: Comment: not explained not explained not explained not explained (except for found, cf. below) Number of found records. This may be used to retrieve the number of records matching a criterion without actually retrieving them (e.g., in conjuction with nrecs=1). For each information field of any of the jobs, one field is included. column name Name of a field in information reply. Only fields which are listed here may appear as jobfield. For each job which information is returned for, one job-tag is present. job Each job-tag will have field-subtags. job/field name="jobKey" row name name="name" name="userGroup" name="deadline" name="constructor" name="destructor" name="state" job's state The fields-tag above lists which field subtags to expect here. The job's key. Present in all replies. All subsequent fields are examples and mey not be present in a particular reply (except for state). The name as given during construction. The list of users as given during construction. The deadline (single human-readable string). User which created the job. User which processes or processed the job. The job's current state. Present in all replies. The list of possible states is as follows: pre-creation creation started application submitted work submitted creation rolled back created assigned finished retrieved closed ok closed failed v v v v v v v v v v v A state starts (but is not equal to!) with a string from the list above. The URL pointing to details as given during construction. not explained name="details" job/action Because this explaination may seem confusing we provide a simple example: – 11 – <?xml version="1.0" encoding="ISO-8859-1" ?> <?xml-stylesheet href="jobs.xsl" type="text/xsl" ?> <jobs> <title>elboss' jobs</title> <user><principal>elboss</principal><in-role>Constructor</in-role></user> <list-options> <option value="byConstructor" selected="1">elboss' jobs</option> </list-options> <fields> <field>jobKey</field> <field>state</field> </fields> <job> <field name="jobKey">4711</field> <field name="state">assigned (destructor elworker)</field> </job> <job> <field name="jobKey">4730</field> <field name="state">creation started</field> <action type="Open"><job-key>4730</job-key></action> </job> </jobs> 2.9 Retrieve result of Job When our job has been finished processing (has reached the state “finished”), results can be retrieved as follows: Relative URL: Method: Parameter: command jobKey command.jsp POST or GET Status: Value: RetrieveJob fixed Comment: required the job's key In case of success, the server replies: Type: Root-tag: Sub-tag: XML job Status: Value: Comment: The full file job.xml (as it was processed by the worker) is returned without surrounding responsetags. The GET-request may therefore be used as an URL pointing to a job's result. 2.10 Close job A job must be closed by the constructor. It is after closing only that a job may be deleted or modified by the constructor. After a job is closed, its result is no longer available to be retrieved! – 12 – Relative URL: Method: Parameter: command stylesheet jobKey state reason command.jsp POST Status: Value: CloseJob fixed Comment: required the job's key required ok or failed required a text specifying why the job failed Only required for state failed. A job should be closed with state failed if no result could be retrieved (eg., it did not arrive in time) or if the result had no acceptable quality to be accepted. A worker will see the reason of closed jobs which he processed. Therefore, you may want to supply a URL pointing to additional info within the reason. Note however, that reason is plain text, not HTML. The purpose of this request is to put the job into a final state. In case of success, the server replies: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... 2.11 Accounting The ErcatoLT server has no special provisions for accounting. However, it is possible to retrieve a list of jobs which have been processed by a particular worker and which are now in closed state. Because the ErcatoLT server restrict deletions1, this list is a reliable way (provided the volume info was supplied correctly in the first place) to compute accounting records if needed. 2.12 Delete job It is after closing only that a job may be deleted or modified by the constructor. Nevertheless, deletions are impossible for a constructor anyway until some moment of time1 after he closed the job. Deletions are done as follows: Relative URL: Method: Parameter: command stylesheet jobKey command.jsp POST Status: Value: DeleteJob fixed Comment: required the job's key The purpose of this request is to release all resources aquired by the job and to delete all its database 1 The actual restriction is a time constraint on how long the job must have been closed. The constraint may be read from the server failure response when trying to delete a job which was freshly closed. A typical value is '10080' (minutes) for 1 week. Accounting information would then have to be compiled every week at a fixed day and time. – 13 – entries. After deletion, a job's name may be used to create a new job. In case of success, the server replies: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... 3. Destructor API The destructor API is not disclosed in the current version of the document. 4. Mediator API Only a subset of the mediator API is disclosed in the current version of the document. An ercato mediator is the rôle to resolve conflicts beween constructors and destructors. Typically, only one user has this rôle whereas multiple users may possess the constructor-rôle. To become a mediator, log in as the mediator user: Username: Paßwort: eladmin <mediator-password-B> (not disclosed) The following activities are described for mediators: 1. 2. 3. 4. 5. 6. 7. Manage users Listing jobs Changing job states Deleting jobs Uploading public client resources Updating the ercato client software release Updating the ercato client look&feel (ftp user elusers) (ftp user elpub) (ftp user elresources) (ftp user elapplication) 4.1 Manage users This is as described in section “2.2 Manage users” on page 4. It is a known limitation of the ErcatoLT server that constructors are allowed to modify the mediator user. In a forthcoming release, only mediators may modify constructors' rights. 4.2 Listing jobs This is as described in “2.8 Retrieve stati of Jobs” on page 9. Other than constructors, mediators will see all jobs and may obtain additional information fields (such as an age field). Only mediators can retrieve information about jobs they did not construct or destruct. – 14 – 4.3 Changing job states A job state can be manually altered by a mediator. This must be done with extreme caution! E.g., after a job is closed, it can manually be put back into the open state. However, most of a job's resources needed in the open state are already gone and attempt to assign and process it will most likely result in a misbehaviour of the ErcatoLT server. You have been warned! Only use this option if you are sure you understand what you are doing... It is relatively safe to manually put a job into the closed state. Relative URL: Method: Parameter: command stylesheet jobKey state command.jsp POST Status: Value: ChangeJob fixed Comment: required the job's key required the job's new state a text specifying why the state was manually altered reason This must be exactly one of the states listed in “list of possible states” in the table on page 11 in section 2.8. A mediator should document manual changes. A destructor (worker) and constructor will see the reason of jobs which he processed. Therefore, you may want to supply a URL pointing to additional info within the reason. Note however, that reason is plain text, not HTML. The purpose of this request is to put the job into a different state. In case of success, the server replies: Type: Root-tag: Sub-tag: job-key state XML response Status: Value: Comment: returned for reliability Any value not starting with “ok” is an error condition! ok: ... 4.4 Deleting jobs This is as described in “2.12 Delete job” on page 13. – 15 – Other than constructors, mediators may delete jobs immediately with no time constraint. Even mediators may only delete jobs in closed state. However, they may manually put a job into closed state before eventually deleting it. Note however, that a forthcoming version could restrict the states that a mediator may manually alter. 4.5 Uploading public client resources Public client resources are public2 resources to be used by the ercato client software which reside on the ErcatoLT server. In some cases, this feature of the ErcatoLT server allows a constructor to spare one server he would otherwise require. The most important example of public client resources are a job's image data. Assuming that an image is called img17.jpg residing in folder 020202a as a public client resource, then the following two rules must be observed: v Within job.xml, note the image's URL as “../../../../pub/020202a/img17.jpg”. v Upload (before the job is opened) and delete (after the job is closed) the image as3: Resource-URL: Paßwort: ftp://[email protected]/020202a/img17.jpg <mediator-password-A> (not disclosed) Apply URLs for different resources according to this scheme. In some networking environments FTP-access is forbidden by active firewall rules, while HTTPaccess is not. Therefore, some mediator rights relying on FTP may depend from where the mediator's computer is located (before or behind a firewall). 4.6 Updating the ercato client software release The ercato client uses client-side binary code which can be updated. Updates are files which are provided by Living Pages Research to the mediator. The most important example of client-side binary code is the java class file archive (JAR file). Assuming that a JAR file is called vericlient.jar, then the following rule must be observed: v Upload the JAR as: JAR-URL: Paßwort: ftp://[email protected]/vericlient.jar <mediator-password-A> (not disclosed) 2 Even though public, an access restriction may be easily enforced by choosing unguessable URLs such as ending in “pub/lfhsklalks0124/img625.jpg”. 3 For FTP-URLs, see “URL must be read as follows” on page 5 in section 2.2. In this example, a folder with name 020202a must first be created (mkdir), stepped into (cd) and deleted using the appropriate FTP client commands. Note, that the user now reads “elpub”. Also note that the MSIE browser has a nasty bug when accessing the same FTP server but with multiple user names from a single machine (try right mouse button 'login as' when you see the wrong directory content and never trust what you see...). Another bug with MSIE is that FTP-sockets are not closed. While the ErcatoLT server closes sockets after 10 minutes of idle time, FTP-accesses may fail meanwhile with a “too many sessions” exception. This is because the ErcatoLT server limits the number of FTP sessions to ten in order to ensure a reliable operation. – 16 – Apply URLs for different binary code according to this scheme. Note that the update becomes effective the next time a worker opens a job for work (the release is checked every time a job is opened by a worker). This applies to to jobs which have been opened before! Be careful not to break existing ercato clients (cf. below). 4.7 Updating the ercato client look&feel The ercato client uses look&feel files which can be updated. Updates are files which are provided by Living Pages Research to the mediator, or are modified by the mediator. Most of those files are HTML files which may be modified as needed. The most important example of a look&feel file is an html file. Assuming that an HTML file is called main.html, then the following rule must be observed: v Upload the HTML as: HTML-URL: Paßwort: ftp://[email protected]/main.html <mediator-password-A> (not disclosed) Apply URLs for different HTML files according to this scheme. Note that the update becomes only effective the next time a new job is opened. The update does not affect existing jobs! Knowing this, a reliabe update strategy can be divised as follows: 1. Download all HTML files (as look&feel) 2. Change all references to vericlient.jar to vericlient-v11.jar (locally) (or whatever release number you wish) 3. Instruct constructors to change client.xsl to refer to vericlient-v11.jar (locally) 4. Upload vericlient.jar as vericlient-v11.jar (as software release) (no interference wtih existing jobs can happen) 5. Upload all locally modified HTML files (as look&feel) 6. Instruct constructors to use modified client.xsl in all subsequently created jobs Now, new jobs will use the new software release while existing jobs continue to use the old software release in peaceful coexistence. When a worker opens a job for working, the ErcatoLT server will actually open file index.html as client software which, in turn, launches main.html. Also note that, the most important file for look&feel, application-client.html can only be altered by constructors as part of job creation (by means of the client stylesheet (referred to as client.xsl). 4.7.1 Useful usage The most useful usage of modification of HTML files is to change the ercato clients overall layout, e.g. as the initial sizes of frames, the orientation of the image (above or besides the data) etc. Note that if you change the number of image viewers, to also change the “childcount” object parameters. The only legal way to changes the size of fields and their orientation (vertical or horizontal) is in – 17 – the application settings (referred to as application.xml). It is the aim that direct manipulation of the HTML files is not necessary and future releases of the ErcatoLT server may discontinue support for such changes. – 18 –
© Copyright 2026 Paperzz