Questionmark Conference 2017
Freeze!
Distinguishing Between Good
Security Policy
and Bad Assumptions
Nikki Eatchel, SVP of Assessment
March 2017
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing
Limited. All other trademarks are acknowledged.
Santa Fe
Questionmark Conference 2017
Santa Fe
Presentation Overview
Review of available security guidelines and protocols
Overview of current security landscape
Review of common security issues
Examine typical security procedures and their impact
(the return on investment)
Resource List
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Guidelines and Protocols
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
International Testing Commission
The ITC Guidelines on the Security of Tests, Examinations, and
Other Assessments (July 2014)
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
The Standards for Educational and Psychological
Testing (AERA, APA, NCME)
Test Administration, Scoring,
Reporting and Interpretation
Rights and Responsibilities of Test
Takers
Rights and Responsibilities of Test
Users
Administration, Scoring and
Reporting of Educational Assessment
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Principles for the Validation and Use of
Personnel Selection Procedures
General Subject Areas






Sources for validity evidence
 Content
 Criterion
Fairness and Bias
Operational Considerations
Appropriate Use
Validation Report
Candidates with Disabilities
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Uniform Guidelines on Employee
Selection Procedures (29 CFR 1607)
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Operational Best Practices for Statewide
Large-Scale Assessment Programs
Chapter 8 Test Security






Development and implementation of security plan
Security of intellectual property
Personnel with access to secure materials
Training materials on security topics
Preventing and dealing with security breaches
Procedures and rules for test administration
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Protocols
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
EDI Centers
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Department of Education: Ohio
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Liverpool Hope University
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
National Healthcare Association
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Current Security Landscape
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape
Security is not an issue that
impacts certain exams,
organizations, or countries.
Security issues are global
and are county and
content agnostic.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape
Just a few examples. . .
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
April 2, 2015
After deliberating for nearly
eight days, the jury found that
the educators, including
former teachers,
administrators, and testing
coordinators, had conspired
to artificially inflate test scores
by changing answers or
guiding students to fill in the
correct responses on a 2009
Georgia Criterion-Referenced
Competency Test, the state’s
annual assessment.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape
Social media and technology
increase security concerns . . .
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Landscape: Globally
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Common Security Issues
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Common Security Concerns
Common security concerns will typically fall into one of the categories below:
Assessment Security Options: Considerations by Delivery Channel and Assessment Model, published by ATP
Security Committee, January 2013, http://www.testpublishers.org/book-store
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Common Security Concerns
Typical student issues:
Assessment Security Options: Considerations by Delivery Channel and Assessment Model, published by ATP
Security Committee, January 2013, http://www.testpublishers.org/book-store
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Common Security Concerns
Typical education/administrator issues:
Assessment Security Options: Considerations by Delivery Channel and Assessment Model, published by ATP
Security Committee, January 2013, http://www.testpublishers.org/book-store
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Procedures:
Development and Design
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Typical Security: Certification/Licensure
There a lot of high tech ways to reduce cheating . . .
…. but these are typically expensive, invasive, and difficult too
implement in an educational environment.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
First Things First
Some foundational areas of assessment development have a substantial impact on
assessment security, and are often overlooked when considering security programs.
Exam Design
Delivery Model
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
First Things First
Common Exam
Design Models
•Linear Fixed Form
•Linear-on-the-Fly
Testing (LOFT)
•Computer Adaptive
Testing (CAT)
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
First Things First
Paper-Based
Typical
Delivery
Channels
ComputerBased
InternetBased
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
First Things First
 Randomization and
Presentation of
Items
 Item Pool Flooding
 Fraud Detection
Items
Security Measures for Consideration
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Randomization
Randomization is a tool that can be applied at various levels:





Exam Categories
Exam Sections
Items Within Sections
Items Across Sections
Item Options
The more unique a test experience is for each test-taker, the less the impact of a single
security breach.
In addition, if a security breach does occur, it is typically easier to identify the source of
the breach.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Randomization
Return on
investment?

Typically reasonable cost
(no increased bank size
needed)

Relatively well
documented, positive
impact on security

No burden on educators,
proctors/invigilators
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Item Pool Flooding
Overwhelm test-takers who want to circumvent the process of studying
by purchasing and memorizing unauthorized exam materials.
 Purpose deceive test-takers by publishing
many more (aligned) items than the
number actually on the assessment (e.g.,
double, triple, or quadruple).
 Individuals will become mentally fatigued
and unable to memorize all questions,
including the intended exam questions.
 Publish questions that don’t align to the
subject of the exam and try to distract or
stump test-takers by forcing them to
memorize irrelevant content.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Item Pool Flooding
Return on investment?

Limited time and nominal cost if content is available

Appears to meet the goal of discouraging the use of
unauthorized content and makes it more difficult for
braindump sites to guarantee the usefulness of the materials
being sold

It is very challenging, if not impossible, to accurately
measure the success of item pool flooding.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Trojan Horse Items
The Trojan Horse (TH) fraud detection item methodology was
developed to
 identify when a test-taker has memorized a test answer key
AND does not truly understand the content of the test
 determine when the test answer key has been exposed by
identifying where, when and which test-taker first shows
knowledge of the answer key
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Trojan Horse Items
Test developers include a statistically significant number of
“extra”, unscored test items (typically six are enough) that are:
 Very easy relative to the content of the exam ( i.e. qualified and
below minimally qualified candidates would be expected to get the
question correct);
 Have both viable, plausible distracters and one or more definitely
correct answer(s); and
 Have an obviously incorrect distracter purposely keyed as the
correct answer.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Trojan Horse Items
The assumption is that test-takers who memorize the answer key
AND do not truly understand the content of the test will:
select the purposely miskeyed Trojan answer because
they do not truly understand very basic concepts; and
will perform well on the test overall because they had
prior access to the answer key.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Trojan Horse Items

Statistical probabilities can be assigned to the combination of the number of TH
items answered with the miskeyed answer option and the overall score on the test
to determine the probability of exam security violation.

These probabilities are put into a matrix and the organization assigns its
“acceptable” threshold for enforcement.
Number of TH questions answered as keyed
1
Test
Score
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
100%
2
3
4
5
6
Probability the test result was NOT a result of exam fraud
Above 1 in 100,000
Between 1 in
100,000 and 1 in 1
million
Between 1 in
100,000 and 1
in 1 million
Between 1 in
100,000 and 1 in 1
million
1 in 1 million and
below
1 in 1 million
and below
1 in 1 million and
below
Assessment Security Options:
Considerations by Delivery
Channel and Assessment Model,
published by ATP Security
Committee, January 2013,
http://www.testpublishers.
org/book-store
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Trojan Horse Items
Return on investment?
 TH items are reliable and accurate in detecting if a test-taker
has had prior access to the exam content AND is not truly
knowledgeable about the exam content
 TH items can be used to detect when an exam answer key has
been compromised.
 Since TH items are not scored and not many of them are
required to use the methodology, TH items have little to no
impact on a test-takers assessment experience.
 TH items are ineffective if the content available to test-takers
reflects the actual correct answer versus the purposely
miskeyed, correct answer.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Procedures:
Policies and Practice
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Policies and Procedures
Mid and high stakes exams require a more structured approach to security, both for the
benefit of the test taker and for the organizations involved in the assessment process.
Basic components for security with high
stake assessments include the following:
 Assessment Security Plans/Security
Handbook
 Security Training
 Response Plan for Security Breaches
 Clearly Delineated Administration
Practices
 Protecting Secure Materials
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Plan/Manual
Prior to
the Exam
During
the Exam
After
the Exam
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Agreement
Sample Security
Agreement
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Training
Step 1 is ensuring that the right policies and procedures are in
place for the assessment.
Step 2 is providing appropriate training for all personnel that is
expected to implement and maintain security procedures
This is the element of security that is often most
neglected, but expecting educational staff to uphold
security procedures without proper training does not
produce the best results.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Breaches: Identification
A Test Security
Manual should
clearly identify the
actions/activities
that are considered
a security breach.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Security Breaches: Notification
In addition to identification, process for notification of a breach is also critical.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Administration Procedures
In addition to a Test Security
Handbook, a Test Administration
Manual is a key component for
strong security.
Standardized administration
policies help reduce testing
anomalies and diminish the
likelihood of perceived or actual
security breaches.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Protecting Secure Materials
Tracking and securing assessment materials is another critical component of a
security program. Even if you deliver assessments on computer, there could still
be assessment material that need to be tracked and securely collected and/or
destroyed.
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Policies and Practice
Typically, establishing, documenting, and training on security
procedures takes some investment of time and resource.
However, the level of effort for these activities is typically a
much smaller investment than the potential revenue loss for
breach of secure content (not to mention the loss of integrity of
the testing program).
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Resource List
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Resource List
•
•
•
•
•
•
•
•
•
•
•
The ITC Guidelines on the Security of Tests, Examinations, and Other Assessments, International
Test Commission (2014)
Standards for Educational and Psychological Testing, AERA, APA, NCME (2014)
Principles for the Validation and Use of Personnel Selection Procedures, Society for Industrial
Organizational Psychology (2003)
Uniform Guidelines on Employee Selection Procedures (29 CFR 1607)
General and Vocational Qualifications: Instructions for Coding Examinations, Joint Council for
Qualifications (2014)
Operational Best Practices for Statewide Large Scale Assessment Programs, Chief State School
Officers and Association of Test Publishers (2013)
Assessment Security Options: Considerations by Delivery Channel and Assessment Model,
Association of Test Publishers (2013)
TILSA Test Security Guidebook: Preventing, Detecting, and Investigating Test Security
Irregularities, CCSSO (2013)
ATP Test Security Messaging: Best Practices Report, Association of Test Publishers (2012)
ATP Test Security Survey Report, Association of Test Publishers (2012)
Report on 2010 DMCA Take-Down Notice Pilot Program, Association of Test Publishers
(2011)
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Questionmark Conference 2017
Santa Fe
Questions?
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.
All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.