Data Processing user test: Test report (DRM/SACM role)
Name:
Executive (EU, EA, NA, JAO):
Operating System and browser used:
Workflow
1) Can you think of any “unhappy” workflows not covered in the workflow document that
should be included?
2) Is the proposed set of workflows good for the particular setup at your executive? If not,
what needs to change?
3) Which workflows from the document did you test? Which worked well, which not so
well?
4) Do you think the proposed workflows using PT/AQUA will make the DRM/SACM tasks
easier and/or reduce the workload (after a transition period to get used to the new
setup)? If not, what do you think is missing?
5) Does the transfer of pipeline products to the APA staging area work well and on
acceptable time-scales?
6) Any other comments on the workflow?
OUS lifecycle transitions
7) Are the automatic lifecycle transitions from AQUA working as they should, i.e. are the
right transitions triggered at the right time?
8) Should any more transitions happen automatically? If yes, which ones, and what should
trigger them?
9) Any other comments on the OUS lifecycle/transitions?
E-mail notifications/Communication
10) Are the e-mail notifications working as detailed in the workflow document?
11) Are you getting the right e-mail notifications, i.e. only those pertaining to your
executive?
12) Is the information in the e-mails sufficient? Do the links in the e-mails work as expected?
13) Does communication via comments in AQUA/PT between DRM and data reducer work
well? If no, what could be improved?
14) Any other comments on the e-mail notifications or comment facility?
Project Tracker interface
15) Is the OUS search interface flexible enough and include all the search criteria you need?
How can the search be improved?
16) Does the interaction between PT and AQUA work as expected, i.e. are state transitions,
edits to the Execution Count, and data reducer assignments made in AQUA reflected in
PT and vice versa?
17) From looking at the OUS status history (at the OUS level node), are you able to reconstruct the OUS history accurately? Can you easily tell when more than one data
reduction loop was done, when an SB needed to go back to the observing queue etc?
18) Can you easily query the MOUS assigned to your data reducer ARC? Can you know what
is the workload of any of the data reducers assigned to you?
19) Is any critical functionality related to the assignment data reducers in PT missing?
20) Is any critical functionality (excluding the caveats listed on the Twiki – we know about
those and they will be implemented in the next release) missing from PT?
21) Any other comments on PT or the DRM tool?
AQUA interface (QA2 tab only)
22) Are the OUS Search and Advanced OUS Search interfaces working correctly? Please
check particularly the search by observing modes.
23) Are the search options sufficient and user-friendly? Do they match your needs? If not,
what would you like to change?
24) Do the different tabs (manual/PL calibration, manual/PL imaging) appear as you would
expect (i.e. only once that processing step has been initialised, and then persist through
Verified)?
25) Is the information given in the different tabs (Summary, Calibration, Imaging) correct?
Did you notice any errors?
26) Summary tab: is there anything missing that you would like to see here? In the case of
pipeline runs can you access the weblog via the link? Do you have all the information
you need to make a QA2 assessment?
27) Manual Calibration/Imaging tabs: does the viewing of uploaded files work well? Is there
any other information you would like to see here?
28) Pipeline Calibration/Imaging tabs: are the pipeline scores displayed? Are the calibrator
source fluxes from pipeline correct? Any other information you would like to see here?
29) ‘Do QA2’ pop-up: is this user-friendly? Do you have all the options and QA2 Fail reasons
needed?
30) Any other comments on AQUA?
Final assessment
a) In your opinion, are PT/DRM Tool/AQUA ready to replace the EPT/SCOPS tracking
system? If no, what CRITICAL functionality is missing?
b) What (if anything) do you see as a major improvement in the PT/AQUA workflow
compared to the previous system?
c) What other functionality would you like?