REMOTE MARKERS SUMMER 2017
Standardisation Briefing
Key Contacts
Type of Query
Whom to Inform
Contact Number/Email Address
Assessment Queries
Team Leader
Details will be provided to you by your Team
Leader
Script Allocations
Assessor Management
[email protected]
Please indicate unit and examiner creditor number as
well as nature of query.
Withdrawing from
Marking
Team Leader and Assessor
Management
[email protected]
Technical issues with
RM Assessor
RM Helpdesk
03303 336 319
Administrative Queries
Related to your
marking
On Screen Marking Helpdesk
01223 553393
Instructions and Forms
Assessor Communications
website
http://www.ocr.org.uk/assessorcommunications/
Please indicate unit and examiner creditor number.
Also inform us whether your withdrawal is for this
session only or permanently.
Regulatory requirements
Examining is regulated by condition H1.1 of the General
Conditions of Recognition. This requires that the awarding
organisation must
•have in place effective arrangements to ensure that, as far as
possible, the criteria against which Learners’ performance will
be differentiated are –
(a) understood by Assessors and accurately applied,
and
(b) applied consistently by Assessors, regardless of the
identity of the Assessor, Learner or Centre.
Purpose of Standardisation
The standardisation process is designed ensure that all
examiners have a well-founded and common understanding of
the requirements of the mark scheme (appropriate to their
responsibilities) and can apply them reliably.
Completing this process enables all examiners to mark
candidates' work consistently and accurately.
It establishes a common standard of marking that should be
used to maintain the quality of marking during the marking
period.
Responsibilities
•The Principal Examiner is responsible for the
professional judgements underpinning the process
of standardisation.
•Team Leader’s are responsible for supporting
examiners through the standardisation process.
Standardisation for Remote Markers(1)
• Question Paper can be accessed from the RM
Portal day after paper sat date.
• SSU (standardisation set-up meeting) take places
7-10 days after paper sat date (this is attended by
Team Leaders and the Principal Examiner).
• Mark scheme is finalised at SSU and practice and
standardisation scripts are selected for the
learning and testing phases for remote markers.
RM Portal
http://support.rm.com/generic.asp?Cref=GP1286643&SrcURL=/ca
Standardisation for Remote Markers(2)
• After the SSU your Team Leader will make contact
with you to inform you that the mark scheme will
be available via RM Assessor(this should be
within 24 hrs of the end of the SSU).
• Once you have accessed the mark scheme you
should login to RM Assessor and access the
practice scripts to begin the learning phase,
followed by the testing phase.
• This process is likely to commence approximately
10-12 days after paper sat date (timescales can be
shorter for later examination papers).
Standardisation for Remote Markers(3)
Exam is Sat
Marker accesses
the Question Paper
SSU Meeting
(Approx 7-10 days
Paper Sat Date )
After SSU - TL
contacts team
Marker can now
access mark
scheme from RM
Assessor
Marker can now
begin the process
RM Assessor
All marking will be carried out on RM Assessor.
URL
https://ca-marking.assessor.rm.com
Standardisation Approval Process (1)
• Download practice scripts, then mark and submit one at a
time. Once you have submitted the first script, view the
definitive mark by accessing the script in your closed items
before continuing with the next practice script.
• Email your Team Leader once you have finished the practice
scripts. Your Team Leader will then contact you and provide
feedback. This is your opportunity to ask any questions
about how the definitive marks were awarded.
• After this discussion has taken place, download the
standardisation scripts. This is your test of marking. Mark
these scripts independently and submit all at once.
• Your Team Leader will then review your scripts and either:
• - approve you to complete live marking
or
• - not approve you and provide feedback. You will then
be asked to complete a second test (STM scripts) which
you will need to download, complete and submit to your
Team Leader.
Standardisation Approval Process (2)
• After second test (if applicable) TL either approves marker for live
marking or informs the marker and the examiner and assessors team
that the examiner will not be progressing to live marking.
Guide
Location
Completing Standardisation Process
On Screen Marking, RM Assessor
Examiner/Specialist Marker, Quick Prompt
Marker guide
On Screen Marking, RM Assessor, Marker Guide
RM Assessor
Documents can be located at
http://www.ocr.org.uk/assessor-communications/
Standardisation Approval Process (3)
Familiarisation and
Learning
Practice Scripts
Testing
Standardisation
Scripts
Approval
Live Marking
Non Approval
Second Test
STM Scripts
Non Approval
Examiner does not
progress to Live
Marking
TL informs
Examiner and
Assessor
Management
Suspension during live marking (1)
• If a Team Leader has concerns regarding your marking (as a result
of your marking on seeded scripts) they will check some live
scripts.
• If following the live marking check your Team Leader still has
concerns they will suspend you on the system and contact you to
inform you that you are suspended from any further marking until
the PE has reviewed the marking.
• The PE will then review the judgements made by your Team Leader
(as soon as possible) and then inform OCR and your Team Leader
of the outcome.
• This could result in the following:
– Your Team Leader makes contact with you to inform you that
you can continue marking or
– An OCR Monitoring and Support Manager contacting you to
inform you that you will not be able to mark any further scripts.
Suspension during live marking (2)
TL carries out
check
Completes live
script checking
form
Check reveals
concerns
TL Suspends
Marker
Sends completed
form to Monitoring
and Support
Manager and PE
Examiner must be
stopped
Monitoring and
Support Manager
contacts Examiner
to inform them of
decision
PE considers
Evidence (24hrs)
Confirms Decision
to Monitoring and
Support
Manager/TL
Examiner should
continue marking
TL informs marker
and removes
suspension
Marker Monitoring
• All examiners are monitored through seeding scripts and
our marker monitoring alert system.
• Where an examiner’s marking is not in line with the rest of
the markers on the panel or their response to a seeding
script is causing concern an alert is triggered.
• Once the alert is triggered the Team Leader is asked to carry
out a check on the live marking of the examiner the alert has
been triggered on.
• Following the check on live marking the team leader either
informs the monitoring team the examiner is fine to continue
or if the Team Leader has
concerns then they will suspend
the marker while the PE considers the evidence.
TL is requested to
carry out a check
This request is made by the Monitoring and Support Team
TL suspends marker on Scoris and advises them of
suspension
TL carries out check
(48hrs)
Completes Alert
Response Form
(ARF)
Check reveals no
concerns
Sends completed
form to Monitoring
and Support Team
Check reveals
concerns
Suspends Marker
If a marker is to be stopped from marking an OCR
Monitoring and Support Manager will make contact.
Examiner must be
stopped
Monitoring and
Support Manager
Contacts Examiner
Confirms to PE/TL
Examiner has been
stopped
Sends completed ARF
to Monitoring and
Support Manager and
PE
PE considers
Evidence (24hrs)
Confirms Decision
to Monitoring and
Support
Manager/TL
Examiner should
continue Marking
TL informs marker
and removes
suspension
Mark Return Dates and Grace Period
Mark Return Dates
• Marks return dates are available on the Assessor
Communications website and will also be
available to you within RM Assessor once you
start marking
Grace Period
• The marking grace period starts at 48 hours.
• It reduces to 12 hours after the 50% marks return
date.
• Reduces to 0 hours after the 100% marks return
date.
Assessor Comms Website (under the heading forms):
http://www.ocr.org.uk/assessor-communications/
Confidentiality of Marking
• Marking must never be carried out in a public place.
• There must be no discussion of work in progress with
anyone other than members of the marking panel and
OCR. Therefore, do not discuss marking with anyone
else including other colleagues or your family.
You should never discuss any aspect of marking or
marking process with the media.
Conflict of Interest (1)
What to do if you download a response and you
think you know the candidate
Although it is very unlikely, there is a remote possibility that
you may recognise that a response you have downloaded for
marking has been written by a candidate who is known to you.
This is a conflict of interest, so if you recognise a response
that you have downloaded, you must reject this immediately.
Where a response you have downloaded for marking has
additional objects, you are advised to check these for any
identifying details before you start marking it, so that you can
reject it immediately if you recognise the candidate.
Conflict of Interest (2)
To reject a response:
1. From the Marking page for the response, click Reject
Response.
A confirmation request is displayed.
2. Click Yes. The response is rejected and the next available
response is displayed.
Conflict of Interest (3)
Please note: this method of rejecting scripts should
only be used where an examiner recognises a
candidate response. For instances such as illegible
scripts or where responses come from an unfamiliar
option route, the appropriate exception process should
be raised.
Marking – Good Practice
• It is important that you mark at a steady rate to
ensure that you are not rushing to complete
your marking to meet the deadlines set.
• It is important to let you team leader know if
you are planning to take any long breaks
during the marking period e.g. not marking any
scripts for a couple of days
Annotation
Purpose and Importance of Annotation
• Provides an audit trail of why marks have been awarded.
• Improves visibility of marking process for centres.
• Allows TLs/PEs to look at AE scripts and understand
where/how marks were awarded. This means useful
feedback to AEs can be given.
Mark scheme
• Follow annotation principles as laid down in the mark
scheme.
• There must be evidence that every page has been seen by
an assessor, including blank pages. Guidance will be
provided in the mark scheme.
Giving Marks (1)
Implications of incorrectly entered marks
• Incorrectly entered marks (e.g. through mis-keying) result in
candidates being issued with the wrong grade. This brings
the integrity of the examination into question.
Ensure that you:
• activate on screen mark confirmation so that when you
enter a mark a large image of the number you have entered
will appear briefly on screen.
• are marking against the right image zone (for optional
questions).
Giving Marks (2)
PLEASE READ CAREFULLY INFORMATION BELOW -THERE
HAS BEEN A CHANGE TO THE GUIDANCE ISSUED IN
PREVIOUS SESSIONS.
Award No Response (NR) if:
• There is nothing written in the answer space
Award Zero ‘0’ if:
• anything is written in the answer space and is not worthy
of credit (this includes text and symbols).
Team Leaders will confirm the correct use of the NR button
with their markers before live marking commences and should
check this when reviewing scripts.
Rubric Errors
PLEASE NOTE
• Where candidates have a choice of questions and
have provided more answers than required, then
all responses are marked and the highest mark
allowable within the rubric is given (by the
system).
• Enter a mark for each question answered into RM
assessor, which will select the highest mark from
those awarded.
This slide is only applicable to papers were there is optionality
Appraisal & Grading of Examiners
At the end of the marking period your Team Leader
will be asked to grade you on the following:
•
Quality of Marking
•
Administration/Communication
This grading will determine whether or not you
require any further training before you are invited to
mark for future sessions.
Payment for Marking
• Payment will only be made for scripts that have
been marked and submitted.
• Payment for marking will be made automatically at
regular intervals during the marking period for
scripts that you have marked and submitted.
Finally………….
Thank you for the marking that you will
be undertaking for OCR for the
June 2017 session.