Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Evaluation System
If you don’t have this tool in your site, you can add it from Site Editor>Edit tools.
Index
Evaluation System .............................................................................. Error! Bookmark not defined.
Accessing the Evaluation System ................................................................................... 2
Working with Evaluation Questions and Headers ........................................................... 3
Working with Templates .................................................................................................. 8
Creating Evaluations ..................................................................................................... 16
Previewing Evaluations ................................................................................................. 25
Changing Evaluations ................................................................................................... 26
Removing Evaluations .................................................................................................. 27
Closing and Reopening Evaluations ............................................................................. 27
Viewing and Working with Evaluation Results............................................................... 28
For Evaluators: Taking an Evaluation ............................................................................ 30
For more information, tips or advice; or if you are experiencing technical problems; please log a
call with ITD Service Desk (ext 2999/2334, or online in NetDesk http://inside/itdservicedesk/).
1
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Introduction
If you’re an instructor or site owner, the Evaluation System tool allows you to conduct anonymous
formative and summative evaluations of your sites and view the results. You may also use this tool to
conduct general surveys.
The Evaluation System tool is quite flexible. Using it, you may:

Create scaled (rating) questions, multiple choice questions with single or multiple answers, and/or freeform-text
questions.

Create evaluation templates, each of which may include questions you have already created, “expert” questions
created by others for your use, and/or new questions that you create during the template creation process.

Create evaluations based on your templates and/or templates that have been created for others for your use.

Choose groups of evaluators for each evaluation, automatically notify these evaluators of upcoming evaluations,
and remind them to complete active evaluations they have not yet taken.

Require central authentication by evaluators or allow them to take evaluations without logging in.

Set the time period during which an evaluation will be available.

View results online for a single evaluation or a series of evaluations conducted during a time period you specify.
(You may access results only after evaluations close.)

Export evaluation results to an Excel spreadsheet, a comma-delineated (CSV) file, or a PDF.

If you have added the Evaluation System tool to multiple sites, access all questions, templates, and evaluations you
have created in one site from any of these other sites. Likewise, you may access results for an evaluation of one
site from another of these sites.
Accessing the Evaluation System
The Evaluation System tool is available in each user’s My Workspace at request. It is not a default tool,
so if you want to access Evaluation System, please log a call through NetDesk
http://inside/itdservicedesk/.
Once the tool is added, to access the Evaluation System tool, click Evaluation System in the tools
menu on the left side of the screen. Sulis displays the Evaluation System home page.
2
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
The layout of the home page (which is also known as the Evaluation Dashboard) varies, depending on
how much information your institution has chosen to display. At a minimum, you will see two lists. On the
left is a list of evaluations to which you have been assigned as an evaluator. For each evaluation in this
section, you see:

the evaluation title.

the name of the course or evaluator group with which the evaluation is associated. If the name is displayed as a
link, this evaluation is available for you to take now.

a note indicating the status of this evaluation (pending=ready for you to take; completed=you have already taken
it).

start and end dates for this evaluation.

the evaluation title.

a note indicating the status of this evaluation (e.g., active, in queue, viewable).

key dates that vary depending on the evaluation’s status.
On the right is a list of your evaluations that are underway or that have recently closed. For each, you
see:
Note: If other view options have been turned on at your institution, you may also see lists of groups you may evaluate and
groups you may be evaluated by.
In addition to the usual tool menu at the top of the screen, embedded links allow you to quickly access
evaluations you need to take, previews of evaluations you are creating or administering, and reports for
your closed evaluations.
To return to this screen from another screen in the tool, either click Evaluations Dashboard in the menu
bar at the top of the screen or click the blue Reset button.
Working with Evaluation Questions and Headers
Questions and headers are the primary building blocks of your evaluations.
Adding Questions and Headers
Tip: You may add questions as you create a template or add questions first and then use them in templates.
3
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Each question or header (for a group of questions) in an evaluation is known as an “item.” To add a
question or header, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Items. Sulis displays the Items screen.
This screen lists items you have already created in this or any other site. For each item, the system displays two
lines of information.
3.
4.

The top line identifies the item as a particular type of question or as a header, provides an indication of the
layout of answer options (vertical or full; for questions only), provides links for activities you may perform on the
item, and displays your name as its creator.

Indented on the second line is the text of the question or header.
In the top right quadrant of the screen, select the type of item you want to add from the pulldown list.
Item Type
Description
Rating scale
A question for which all possible answers are determined by the rating scale you
select
Multiple choice
A question for which you specify multiple possible answers, only one of which may
be selected by each respondent
Multiple answer
A question for which you specify multiple possible answers, any or all of which
may be selected by each respondent
Free text/essay
question
A question that requests a text response; you specify the number of lines allowed
in the response
Text header
Text that introduces a new section of the evaluation
Click Add Item. Sulis displays the Add/Edit Item screen for the type of item you chose.
4
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
The fields of this screen vary somewhat, depending on the type of item you are adding.
5.
Enter the text of the new question or header in the Item Text box. Use the editor’s pulldown lists and buttons to
format the text as appropriate. For information on using these options, see Using the Rich Text Editor.
6.
For a rating scale question, select the scale you want to use from the Scale pulldown list.
7.
For a multiple choice or multiple answer question, enter answer options in the Enter choices section of the screen.
8.

To delete an existing option, click the Remove button for it.

To add a new option, click Add Scale Point. Sulis displays a blank field in which you enter appropriate
text.
In the Display Setting Hints section, select options to control the display of the question and its answers.

For a rating scale, multiple choice, or multiple answer question only, choose the display you prefer from the
Scale Display Setting pulldown list.
Option
Example
Compact (available
for rating scale
questions only)
5
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Option
Example
Compact colored
(available for rating
scale questions only)
Full
Full colored (available
for rating scale
questions only)
Stepped (available for
rating scale questions
only)
Stepped colored
(available for rating
scale questions only)
Vertical

For a free text/essay question only, select the maximum number of lines allowed from the Response size
pulldown list.

To add the answer option N/A (not applicable) to the possible answers for any question, click the N/A (not
applicable) box.

To allow respondents to enter explanatory text in response to a rating scale, multiple choice, or multiple
answer question, check Include comment box.

Choose an appropriate category for the question or header. If this question rates the evaluate(s) or
instructor(s), click Evaluatee/Instructor; otherwise, be sure that the default setting of Group/Course is
selected.
Note: When an evaluation pertains to multiple instructors, the instrument contains a subhead for each evaluatee. All
Evaluatee/Instructor questions are repeated under each instructor’s subhead.
9.
To save your work, click Save item. Sulis displays the Items screen, which now includes your new item. Or, to
return to the Items screen without saving your work, click Cancel.
Previewing Questions and Headers
You may preview any item (that is, any question or header) listed on your My Items screen. To do so,
access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Items. Sulis displays the Items screen.
3.
Find the item you want to preview, and click the Preview link for it. Sulis displays the Preview Item screen. It
shows how the header or question and answer will display to respondents.
4.
Return to the Items screen by clicking Close Window.
6
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Editing Questions and Headers
Note: When you change a question or header listed in My Items, your change appears only in the item in the My Items
screen, “master” template(s) listed on the My Templates screen that use this item, and evaluations you create from these
templates in the future. The system does not apply the change to the question in pre-existing evaluations. To change items in
an existing evaluation, see Changing Evaluations.
To change a question or header, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Items. Sulis displays the Items screen.
3.
Find the item you want to change, and click the Edit link for it. Sulis displays the Add/Edit Item screen.
4.
Proceed as instructed in Adding Questions and Headers, steps 5-9, making changes as needed.
Copying Questions and Headers
When you want to create a new question or header, you may want to copy an existing item and then edit
the copy. To copy a question or header, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Items. Sulis displays the Items screen.
3.
Find the item you want to copy, and click either of the two Copy links for it. Sulis displays the copied question or
header at the bottom of the list of items.
Deleting Questions and Headers
Note: When you delete an item, it is not available for use in future templates, but it continues to be available in templates that
already contain it. Consequently, the deleted item still exists in all evaluations based on templates that you made that
contained the item, even though the item is no longer available for use in new templates.
To remove a question or header from your evaluation items list, access the Evaluation System tool and
follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Items. Sulis displays the Items screen.
3.
Find the item you want to delete, and click the Remove link for it. Sulis displays a confirmation screen.
4.
To proceed with the deletion, click Remove Item. Sulis displays the Items screen, and the item you chose is no
longer listed. Or, to return to this screen without removing the item, click Cancel.
7
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Working with Templates
In the Evaluation System tool, each evaluation you create is based on an evaluation template. Having
templates helps you save time because you can create many evaluations from a single template and
then change each individual evaluation as needed without affecting the “master” template. You may
create as many different “master” templates as you like. Depending on how your system has been set
up, you may be allowed work with templates you’ve created and/or with templates created by others for
your use.
Adding Templates
To create a template on which to base evaluations, access the Evaluation System tool and follow these
steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Templates. Sulis displays the Evaluation Templates screen.
This screen lists templates you have already created in this site and any other site you own along with templates
created by others for your use. For each template, the system displays its title, the name of the template’s owner,
the date it was last updated, and links for the activities you may perform on the template.
3.
Click Add Template in the upper right quadrant of the screen. Sulis displays the Edit Template Title/Desc.
screen.
4.
Enter a title for the template in the Title field. An entry in this field is required
5.
Enter explanatory text in the Description field, if desired.
6.
To proceed, click Save. Sulis displays the Edit Template screen. (Or to return to the Evaluation Templates screen
without creating the template, click Cancel.)
8
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
7.
If desired, add one or more existing questions and/or headers to the template as follows:
a. Select the Existing option from the Add Item dropdown list and click Add. Sulis displays
the Choose existing items screen.
This screen lists all of the questions and headers available for use in your templates.
b. To search for a specific item, enter a keyword into the box in the upper right and click
Search.
c. Check the box for each item you would like to add.
d. To add the selected items to the template, click the Insert Items button at the top or bottom
of the list. Sulis displays the Edit Template screen, and the selected items have been added
at the bottom of the list. Or, to return to the previous screen without adding items, click
Cancel.
8.
If desired, add one or more new questions and/or headers according to the instructions in Adding Questions and
Headers, steps 3-9.
9.
If desired, add one or more expert questions as follows:
a. Select the Expert option from the Add Item dropdown list and click Add. Sulis displays the
Expert Items screen.
9
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
The top part of this screen shows the steps in selecting expert items. The lower part of the screen contains
links to categories in which expert questions are available.
b. Click the name of the desired category. Sulis displays one of two screens:
o
If the category is further subdivided into objectives, Sulis displays the links to the objectives for the
category.
o
If the category contains no further subdivision, Sulis displays a list of expert questions within the category.
10
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
c. If Sulis is displaying a list of objectives, click the name of the objective for the question you
want. Sulis displays a list of questions within that objective.
d. Check the box for each question you want to add.
e. To add the selected items to the template, click the Insert Items button at the top or bottom
of the list. Sulis displays the Edit Template screen, and the selected items have been added
at the bottom of the list. Or, to return to that screen without adding items, click Cancel.
10. If desired, create blocks of questions that use the same rating scale and display format. To create each block:
a. Click the white box for one of the rating scale items.
11
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Sulis displays an asterisk ( ) next to each item with the same rating scale and display style. These are the
only items you may add to this block.
b. Check boxes for each additional item you want to add to the group. You may select one,
some, or all of the available items. After you click the first one, the Create Block button at
the bottom of the list becomes active.
c. Click Create Block. Sulis displays the Edit Block screen.
12
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
At the top of the screen, the system displays the items you’ve chosen to group together. It also indicates the
type of scale for the questions.
d. Rearrange the items in the block as needed by dragging and dropping them.
e. If desired, in the Item Header Text box, enter instructions or explanatory text that applies to
this block of questions. Use the editor’s pulldown lists and buttons to format the text as
appropriate. For information on using these options, see Using the Rich Text Editor.
f.
Select appropriate display options and the category for this block of questions.
g. To complete your work, click Save item. Sulis displays the Edit Template screen. The
selected questions are now grouped into a unit. (Or, to return to that screen without adding
the block, click Cancel.)
11. Work with all items in the template as needed.
13
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
You have these options:

To preview any item, click the Preview link for it. To return to the preceding screen, click Close Window.

To change an item, click the Edit link for it and proceed as instructed in Adding Questions and Headers, steps
5-9.

To remove an item from the template, click the Remove link for it. When Sulis displays a confirmation
screen, click Remove Item.

To rearrange items, either drag and drop them in the preferred order or use the numbered pulldown lists to
change the item’s number in the list. To save the order you have established, click Save Order. To revert to
the original order, click Revert Order.
Note: You cannot add new items to an existing block. Instead, unblock (see below) and then recreate the block to include all
appropriate items.

To “unblock” items in a block, click the Edit link for it. When Sulis displays the Edit Block screen, click the
Remove link in the top right quadrant of the screen. When Sulis displays a confirmation screen, click
Remove Item. Sulis unblocks all items in the group but does not remove the items from the template.
Tip: After creating a template, you may create an evaluation based on it by clicking the New Evaluation link to the right of
the template title.
Previewing Templates
You may preview any template listed on your Evaluation Templates screen. To do so, access the
Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Templates. Sulis displays the Evaluation Templates screen.
3.
Find the template you want to preview, and click the Preview link for it. Sulis displays the Preview Item screen. It
shows how an evaluation based on this template will display to respondents.
4.
Return to the Evaluation Templates screen by closing the browser window.
14
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Editing Templates
Note: When you edit a template as explained below, you are changing only the “master” template (that is, the template you
base future evaluations on). The system does not apply your changes to the copy of the template used in any evaluation(s)
you have already created. To change the copy of the template used in any particular pre-existing evaluation, see general
information in Changing Evaluations and instructions in Specifying Settings and Dates.
You are not permitted to edit templates created by others.
To change a question or header, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Templates. Sulis displays the Evaluation Templates screen.
3.
Find the template you want to change, and click the Edit link for it. Sulis displays the Edit Template screen.
4.
To change the name or description, click the Edit link to the right of the Template title, make changes as needed,
and click Save. Sulis redisplays the Edit Template screen.
5.
To work with the items in the template, proceed as instructed in Adding a Template, steps 7-11.
Copying Templates
When you want to create a new template, you may want to copy an existing template and then edit the
copy. To copy a template, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Templates. Sulis displays the Evaluation Templates screen.
3.
Find the template you want to copy, and click the Copy link for it. Sulis displays the copied template at the bottom
of the list.
Deleting Templates
Note: When you remove a template, it is no longer available for you to use in new evaluations. The template persists,
however, in evaluations you have already created based on it.
You are not permitted to delete templates created by others.
To remove a template from your evaluation items list, access the Evaluation System tool and follow
these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Templates. Sulis displays the Evaluation Templates screen.
3.
Find the template you want to delete, and click the Remove link for it. Sulis displays a confirmation screen.
4.
To proceed with the deletion, click Remove Template. Sulis displays the preceding screen, and the template
you chose is no longer listed. Or, to return to this screen without removing the template, click Cancel.
15
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Creating Evaluations
The process of creating an evaluation involves three steps:
1.
Adding the Evaluation.
2.
Specifying Settings and Dates.
3.
Assigning Groups of Evaluators.
You do not need to perform all of these steps during one session. If you exit before finishing step two or
three, however, the evaluation will be partially complete. You then have 5 calendar days to complete it
before it is automatically deleted.
Adding the Evaluation
Note: Evaluations are based on templates, so at least one evaluation template must be available for your use (created by you
or someone else) before you can create an evaluation.
To create a new evaluation, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Evaluations. Sulis displays the My Evaluations screen.
This screen contains lists of your evaluations in all sites you own. Each list includes evaluations in a different state.
Evaluation State
Partially Complete
Explanation
Evaluations you began creating but did not complete, because you exited either
step 2 (Specifying Settings and Dates) or step 3 (Assigning Groups of Evaluators)
16
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Evaluation State
Explanation
before finishing. Warning: If you do not complete a partial evaluation within 5
calendar days, the system automatically deletes it.
In-Queue
Evaluations that are scheduled to begin but have not yet started.
Active
Evaluations that have been released to evaluators; evaluations are not yet due.
Users can take these evaluations now.
Closed
Evaluations whose due date has passed. Users cannot take these evaluations any
longer.
3.
In the upper right quadrant of the screen, click Add Evaluation. Sulis displays the Add Evaluation screen.
4.
In the Title field, enter a name for this evaluation.
5.
In the Instructions box, provide instructions to the users. Use the editor’s pulldown lists and buttons to format
the instructions as appropriate. For information on using these options, see Using the Rich Text Editor.
6.
In the lower portion of the screen, click to choose the template you want to use.
7.
To proceed, click Continue to Settings. Sulis creates a “partial” evaluation and displays the Evaluation
Settings screen for it. (Or, to return to the preceding screen without creating the evaluation, click Cancel.)
8.
To go on to the next step in the overall process, see Specifying Settings and Dates. (Otherwise, return to the My
Evaluations screen by clicking Cancel. The saved evaluation is displayed in the list of partially complete
evaluations.)
Specifying Settings and Dates
You may specify settings and dates for an evaluation at either of two points:

As step two of creating a new evaluation (see Creating Evaluations), immediately after completing step one (see
Adding the Evaluation), or
17
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017

any time within 5 calendar days of exiting the creation process without completing the evaluation. In this case, you
access the evaluation by clicking the Continue creating this evaluation link for it in the Partially Complete
Evaluations section of the My Evaluations screen.
Warning: If you do not complete a partial evaluation within 5 calendar days, the system deletes it.
In either case, the system displays the Evaluation Settings screen.
18
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Proceed as follows:
9.
In the Evaluation Title section, make any changes you like:

Edit the evaluation’s title and/or instructions by changing the entries in those fields.

View the selected template by clicking Preview and closing the window to return to this screen.
19
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017

Change the template by clicking Edit and proceeding as directed in Editing Templates, beginning with step 4.
Note: When you edit the template in an evaluation, you are changing only a copy of the template (the copy used in this
evaluation). The system does not apply these changes to the “master” template (the template you see on the My Templates
screen and use to create additional evaluations). To change the “master” template, proceed as directed in Editing Templates.
10. In the Evaluation Dates section, enter appropriate start and end dates for this evaluation or select dates via the
calendar icons. Your institution may also ask that you provide a separate stop date to allow users a grace period
beyond the due date. Users cannot access the evaluation before the start date or after the end date.
11. In the Evaluation Results section:
a. Specify who can view the evaluation results after the evaluation has closed. The options are
as follows:
o
Private (only you may see the results)
o
Configurable (view options are pre-set by system administrator)
o
Public (all users may see the results)
b. If the screen contains a View Date field, specify the date on which results are to be made
available. If you do not have or use this field, the evaluation is automatically made viewable
on its due date. Review the default settings established by your system administrator. If any
of these settings need to be changed, contact the administrator.
12. In the Participant Completion Settings section, review the default settings established by your system administrator.
If any of these settings need to be changed, contact the administrator.
13. In the Administrative Settings section, use the dropdown list to specify whether users need to log in to participate in
the evaluation. The options are:

Central Authentication Required (only SULIS users may participate in this evaluation, and they must
log into Sulis to do so)
Important:
If you choose Central Authentication Required, all evaluators must log into Sulis in order to
take this evaluation. Users who do not have a SULIS username and password and do not belong to an
evaluator group will not be able to access the evaluation.

None (anyone with access to the evaluation’s URL may participate in the evaluation, and they need not log
into Sulis to do so)
14. In the Evaluations Notification section:
a. To view the notification or reminder email sent to participants, click the appropriate
View/Edit… email? link. For information on changing the content of these emails, see
Changing Notification Emails.
b. Select from the dropdown list to specify how frequently reminder emails are to be sent to
evaluators who have not yet responded.
c. In the Email From/Reply-to Address field, enter the email address that will be used as
both the “from” and “reply to” addresses in these emails.
15. To continue, click Continue to Assign to Courses. Sulis displays the Edit Evaluation Assignment screen.
(Or, to return to the previous screen without saving your work here, click Cancel.)
16. To proceed to the next step in the overall process, see Assigning Groups of Evaluators.
Changing Notification Emails
20
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Sulis automatically sends an email to evaluators when your evaluation becomes active. Subsequently, if
the Authentication control setting for this evaluation is set to Central Authentication Required, the
system also automatically sends a reminder email to evaluators who have not yet responded at intervals
you specified in step 13 of Creating Evaluations.
Each email contains default text and appropriate information specific to your evaluation such as
evaluation due date, the name of the evaluation, and its URL.
Since these emails are system-generated and contain the information your evaluators need, there is
usually no need for you to change them. If desired, however, you may change the default text and key
values—that is, the values that represent your system- and evaluation-specific information for either or
both emails. To do so, when you reach step 13 of Creating Evaluations, proceed as follows:
17. Click the link for View/Edit the notification email? or View/Edit the reminder email?, depending on
which email you want to change. Sulis displays a screen made up of two sections. The top section contains a list of
key values you may want to use in your email.
The bottom half of the email shows the default text with the key values inserted in their default locations.
21
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
18. To change the email, click Edit this Email Template in the bottom half of the window. Sulis displays the Edit
Available Email Template screen. (Or, to return to the Evaluations Settings screen without making changes, click
Close Window.)
The top half Edit Available Email Template screen repeats the list of key values from the preceding screen. The
bottom half contains editable text fields for your email.
19. Make changes in the subject line and body of the message as needed.
22
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
20. To save your work, click Save Changes and then return to the Evaluation Settings screen by clicking Close
Window. (Or, to return to that screen without making changes, click Close Window.)
Assigning Groups of Evaluators
Note: After an evaluation has become active, you may not de-select, change, or add groups of evaluators.
You may assign all authorized participants in one or more sites to take your evaluation. You also have
the option of creating and assigning ad hoc groups to take the evaluation. You may choose these groups
of evaluators at either of two points:

As step three of creating a new evaluation (see Creating Evaluations), immediately after completing step two (see
Specifying Settings and Dates), or

any time within 5 calendar days of exiting the creation process without completing the evaluation. In this case,
access the evaluation by clicking the Continue creating this evaluation link for it in the Partially Complete
Evaluations section of the My Evaluations screen. At the bottom of the Evaluation Settings screen, click Continue
to Assign to Courses.
Warning: If you do not complete a partial evaluation within 5 calendar days, the system deletes it.
In either case, the system displays the Edit Evaluation Assignment screen.
Proceed as follows:
1.
To see all available groups, click both blue arrows. Sulis displays two lists.
23
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
2.
If needed, create a new ad hoc group as follows:
a. Click Create a new Adhoc Group now. Sulis displays the Creating a New Adhoc Group
screen.
b. Enter a descriptive name for this group in the Group Name field.
c. In the large white box, enter either Sulis username or email address of each person you
want to include in the group. Enter each name on a separate line.
d. Click Create Adhoc Group. Sulis displays Modifying Adhoc Group screen. (Or to return to
the previous screen without adding this group, click Back to Evaluation Group
Assignments.)
24
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
e. To delete any of the users you have added, click the X button by the name(s).
f.
To add more users, proceed as in steps 2c and 2d.
g. Return to the Edit Evaluation Assignment screen by clicking Back to Evaluation Group
Assignments.
3. Select the group or groups of evaluators you want to assign this evaluation to by clicking the box
for each appropriate group.
Note: You may need to reopen the expanded lists of groups by clicking the blue arrows again.
4.
To proceed, click Save Assigned Groups. Sulis displays the Evaluation Assignment screen.
(Or to return to the preceding screen, click Back.)
This screen presents a summary of the dates and groups you have selected.
5.
To complete your work, click Done. Or to change the group assignments, click Change Assigned Groups
and repeat steps 1-4.
Previewing Evaluations
To preview an evaluation, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
25
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
2.
In the section entitled Evaluations I Am Currently Creating or Administering, find the evaluation you want and click
its name.
Tip: You may also preview an in-queue, active, or closed evaluation from the My Evaluations screen. Simply click the name
of the evaluation you want to preview.
3.
To return to the Evaluation System home page, click the Reset button.
Changing Evaluations
The data you can change in an evaluation depends on the state of the evaluation (partially complete, inqueue, active, or closed).
Evaluation State
Changes Allowed/Prevented
Partially Complete
You may add, delete, or change any information in a partial evaluation (name,
description, template, delivery dates and other settings, and assigned groups).
Warning: If you do not complete a partial evaluation within 5 calendar days, the
system automatically deletes it.
In-Queue
You may change any information in an in-queue evaluation (name, description,
template, delivery dates and other settings, and assigned groups).
Active
You are not permitted to change assigned groups for an active evaluation. You
may change some evaluation settings, but you may not change the evaluation’s
start date, email template, or authentication control setting.
Closed
The only changes you may make to a closed evaluation are those that pertain to
evaluation results sharing and notification/reminder emails.
Note: For information about completing a partial evaluation, click the Continue creating this evaluation link for it and
proceed as instructed in Specifying Settings and Dates and Assigning Groups of Evaluators.
Changing Evaluation Settings
To edit settings for an in-queue, active, or closed evaluation, access the Evaluation System tool and
follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Evaluations. Sulis displays the My Evaluations screen.
3.
Find the evaluation you want and click the Edit link for it. Sulis displays the Evaluation Settings screen.
4.
Make changes as needed (see Specifying Settings and Dates, steps 1-6).
5.
To save your work, click Save Settings. Sulis saves your changes and displays the My Evaluations screen
again. (Or to return to that screen without making changes, click Cancel.)
Changing Assigned Groups
Note: You may add and remove group assignments and change the makeup of assigned groups before an evaluation
becomes active. After it has become active, however, you may not modify assigned groups in any way.
To edit group assignments, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Evaluations. Sulis displays the My Evaluations screen.
26
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
link for the assigned group. Sulis displays the
3.
Find the evaluation you want and, in the Assigned column, click the
Display Evaluation Assignments screen.
4.
Click Modify assigned groups for this evaluation and proceed as instructed in Assigning Groups of
Evaluators.
Removing Evaluations
Note: You cannot remove an evaluation that has responses.
To remove an evaluation, access the Evaluation System tool and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Evaluations. Sulis displays the My Evaluations screen.
3.
Find the evaluation you want and click the Remove link for it. Sulis displays the Remove confirmation screen.
4.
To proceed, click Remove Evaluation. Sulis displays the My Evaluations screen. (Or, to return to that screen
without removing the evaluation, click Cancel.)
Closing and Reopening Evaluations
Closing Evaluations Early
You may close any active evaluation before its due date by changing the due date (see Changing
Evaluation Settings). The system also allows you to close an evaluation immediately. To close an
evaluation right now, access the Evaluation System and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Evaluations. Sulis displays the My Evaluations screen.
27
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
3.
In the Active Evaluations list, find the evaluation you want to close and click the
closes the evaluation immediately and moves it to the Closed Evaluations list.
Close Now link for it. Sulis
Re-Opening Evaluations
The Evaluation System tool allows you to re-open a closed evaluation. To do so, access the Evaluation
System and follow these steps:
1.
If you are not on the Evaluation System home page, click the Reset button to display it.
2.
In the menu bar at the top of the screen, click My Evaluations. Sulis displays the My Evaluations screen.
3.
Find the evaluation you want to open and click the Re-open link for it. Sulis displays the Evaluation Settings
screen.
4.
Change the Due Date entry to the preferred due date for this evaluation.
5.
Click Reopen and Save Settings. Sulis displays the My Evaluations screen and the evaluation is now in the
list of active evaluations. (Or, to return to that screen without reopening the evaluation, click Cancel.)
Viewing and Working with Evaluation Results
After an evaluation ends, the system sends an email that provides a link to a report containing evaluation
results.
To view the results, follow these steps:
1.
Access the report in any of these ways:

To access it from the notification email you received, click the report link provided, or proceed as directed in
the Alternate Instructions section at the bottom of the email.

Log into Sulis. From any site, access the Evaluation System tool by selecting it from the tools menu on the left
side of the screen, and click the View Report link for the appropriate evaluation.

If you are already in Sulis and are on the My Evaluations screen, click the Report link for the appropriate
evaluation (listed under Closed Evaluations).

If you have been given the direct link for the report, paste it into the address bar of your browser. Depending
on the settings for the evaluation, you may need to log into Sulis before viewing the report.

If you are already in Sulis and are on the My Evaluations screen, click the Report link for the appropriate
evaluation (listed under Closed Evaluations).
If the evaluation was assigned to more than one group, Sulis displays the Choose Report Groups screen.
2.
If the system is displaying the Choose Report Groups screen, check the box(es) for the group(s) for which you want
to display results and click Submit. Sulis displays the report.
28
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
The report shows results for the course/group items first, followed by questions pertaining to the instructor/evaluate.
If multiple instructors are being evaluated, the report contains an identical set of questions for each of them.
For each question, you see the total number of responses, the number of responses received by each answer
option, and the percentage of the total represented by the answer count for each option.
If the evaluation contains essay/text answer questions, all submitted answers are listed beneath each question.
29
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
Changing the Display
Links near the top of the screen allow you to filter out or select for the answers to essay questions.

To remove essay questions and their answers from the display of the report, click Hide All Text Responses.

To include these questions and answers again, click View All Text Responses.

To view only the essay answers, click View Written Responses Only.

To return to the display of all questions and answers, click View All Items.
Exporting results
You may export report raw data to a CSV (comma-delimited-values) file or an Excel spreadsheet on your
computer. You may also export the formatted report as a PDF.
Exporting Raw Data
To export raw data to a CSV file, display the report and follow these steps:
1.
In the upper right corner of the screen, click the Export CSV link.
2.
Proceed as directed in Opening or Downloading an Attachment or Other File.
3.
In the upper right corner of the screen, right-click the Export Excel link.
4.
Proceed as directed in Saving an Attachment or Other File to Your Computer.
To export raw data to an Excel spreadsheet on your desktop, display the report and follow these steps:
Exporting the Formatted Report
To export the formatted report as a PDF, display the report and follow these steps:
5.
In the upper right corner of the screen, click the Export PDF link.
6.
When Sulis displays the PDF, save it to your computer.
For Evaluators: Taking an Evaluation
To take an evaluation, follow these steps:
1.
Access the evaluation in any of these ways:

To access it from the notification email you received, either click the evaluation link provided or proceed as
directed in the Alternate Instructions section at the bottom of the email.

Log into Sulis. From any site, access the Evaluation System tool by selecting it from the tools menu on the left
side of the screen. Click the linked group or site name associated with the evaluation you want to take.

If you have been given the direct link or the permalink for the evaluation, paste this URL into the Address bar
of your browser. Depending on the settings for the evaluation, you may need to log into Sulis before viewing
the evaluation.
Sulis displays the evaluation.
30
Sulis Support Documentation
Adapted from rSmart SULIS User Guide for version 2.7
Centre for Teaching and Learning, University of Limerick 13/07/2017
The layout, number, and display of questions and answer options and labels may vary greatly from one evaluation
to another.
2.
If the evaluation includes a Switch Group dropdown list under the name of the evaluation, you belong to two
groups of evaluators and may take this evaluation as a member of either group. To change groups, select from the
dropdown list and click the Switch button.
3.
Following instructions (if any), click to select an answer to each question. (Depending on how the evaluation was
set up, you may not be allowed to skip questions.)
4.
Click Submit Evaluation. Sulis submits the evaluation and changes your status for this evaluation to
completed.
31