Welcome
Workshop Goals:

Understand data sources

Learn the basic components of an effective
assessment

Analyze and effectively use data
Overview of NOCTI
 Over 40 years serving the CTE community

Non-profit organization

Owned and operated by the 50 states

Works with both secondary and post-secondary
populations

The Whitener Group (a NOCTI subsidiary) provides
services to industry
NOCTI Customer Breakdown
Industry Connections
NOCTI Products
Assessments

Over 80 Job Ready tests
offered

30 Pathway tests in 10
clusters available

Over 90 private industry
employment and/or
advancement tests
delivered

20 teacher-level
assessments
Supplemental Products
 Study guides for all Job Ready
and Pathway assessments
 Countdown Kits offered for 40
titles
 Successful Schools-Successful
Programs Guide
NOCTI Job Ready Assessments

Assess technical skills at the occupation level (e.g.,
Accounting, Carpentry, Pre-Engineering)

Measure aspects of occupational competence such
as factual and theoretical knowledge

Target secondary and post-secondary program
completers
Assessment Blueprints

Blueprints include:

Competency List

Content
Distribution

Sample Items and
Jobs
NOCTI Job Ready Assessment Blueprint
Number of Questions: 222
Areas Covered:
Accounting
Assessment Blueprints to Prepare
How can I prepare my students for the NOCTI assessment?
Assessment Blueprints!!!

Ideas for using blueprints in the classroom:

Use throughout the school year as a syllabus and guide
for review

Involve the students: have them calculate the number
of questions in each content area

Send a copy home with students to share with parent
Assessment Results
 NOCTI’s Standard Score Reporting Package:

Individual Score Reports

Group Score Reports

Comparative Data
 Other Reports Available:

Task-Link Reports

Integrated Academic Reports
NOCTI Standard Score Reports
Individual Score
Reports
• Provided for each
test taker
• Scores by duty and
total
• Pre-/post-test
comparisons
• Comparative data
NOCTI Standard Score Reports
Group Score Report
• Provided for the
testing group
• Duty and total
scores for each
participant
NOCTI Standard Score Reports
Analysis of Scores
Report
• Provided for the
testing group
• Comparative data
• Criterionreferenced cut
score
• Other statistical
information
Using Data to Improve Instruction
Using Data to Improve Instruction
Collect Data:
Become familiar with
your data
Understanding Your Data
 National
 State
 District/System
 School
 Program
 Teacher
 Class
 Student
 Standards
What the Data Reveal
Achievement gaps
Differences between subgroups
Trends
Progress
Assessment
 Includes process of collecting, analyzing, and
interpreting information
 Determines the degree to which students are
becoming competent and meeting standards
 Occurs at many levels (e.g., classroom and program
assessments to state and national assessments)
Good Assessments
 Valid
 Test items measure what they are supposed to
measure: the important content or concepts
that were taught and that the student must
know
 Test is relevant and based on current standards
 Subject matter experts develop and evaluate
test items
Good Assessments
 Reliable
 Test and its items produce a similar
score over time
 Scores are consistent, not contradictory
 Scores are not affected by chance
Student Assessment
 On-going with frequent opportunities for feedback
 Use a variety of different types of assessment (e.g.,
written and performance, formative and
summative) during the year
Summative Assessment
 Pretests (covering all major content areas)
are used to identify entry skills and measure gain
 End-of-program or annual test
 Broader in scope, for comparison of groups
Formative Assessment
 Continuous correction from quizzes, homework,
projects, participation
 Use practice items for instructional revision
before summative assessment
Building an Assessment
 Test specification matrix or “Blueprint”





National Standards
Duties and Tasks
Written/Performance
Weighting
Panel of Experts
Step 2
Analyze Data:
Draw conclusions from data
Classroom Example
A Carpentry
Class
Areas on a NOCTI Carpentry Test
Safety
Tools and Accessories
Blueprint Reading and Estimation
Foundations, Forms and Concrete
Rough Framing
Exterior Finish
Interior Systems Installation
Interior Finish
Carpentry Related Mathematics
Individual Carpentry Pre-Test
Pre-test % Correct
Student 1
Post-test % Correct
Student 1
Safety
54.6
--
Tools and Accessories
30.0
--
Blueprint Reading and Estimation
41.1
--
Foundations, Forms, and Concrete
36.4
--
Rough Framing
53.8
--
Exterior Finish
36.4
--
Interior Systems Installation
40.0
--
Interior Finish
27.3
--
Carpentry Related Mathematics
35.9
--
Carpentry Pre-test Areas
Total
40.0
Carpentry Class Pre-Test
Carpentry Pre-test
Areas
Student 1
% Correct
Student 2 Student 3 Student 4
% Correct % Correct % Correct
Student 5
% Correct
Student 6
% Correct
Class
Average
% Correct
54.6
45.5
36.4
27.3
27.3
54.5
40.9
30.0
20.0
10.0
20.0
10.0
30.0
20.0
41.1
17.6
23.5
23.5
29.4
35.3
28.4
36.4
54.5
45.5
45.5
36.4
54.5
45.5
Rough Framing
53.8
38.5
30.8
38.5
30.8
53.8
41.0
Exterior Finish
36.4
36.4
27.3
36.4
36.4
45.5
36.4
Interior Systems
Installation
40.0
60.0
40.0
60.0
60.0
80.0
56.7
Interior Finish
27.3
9.1
18.2
27.3
9.1
27.3
19.7
Carpentry Related
Mathematics
35.9
45.5
27.3
36.4
36.4
54.5
39.4
40.0
34.0
28.0
33.0
29.0
46.0
35.0
Safety
Tools and
Accessories
Blueprint Reading
and Estimation
Foundations, Forms,
and Concrete
Total
Carpentry Class Post-Test
Carpentry Post-test
Areas
Participant
Group
Site
State
Nation
Safety
100.0
86.4
79.5
72.5
75.5
Tools and Accessories
91.7
79.2
78.2
70.7
70.6
100.0
95.0
83.8
64.3
66.4
92.3
69.2
65.8
51.6
52.9
Rough Framing
97.4
71.8
69.8
63.1
64.1
Exterior Finish
91.7
64.6
67.6
58.1
57.8
Interior Systems Installation
87.5
53.1
61.9
55.6
56.5
Interior Finish
87.5
79.7
75.6
66.6
67.0
Carpentry Related
Mathematics
100.0
56.3
68.2
66.1
67.0
94.3
72.0
71.4
62.7
63.4
Blueprint Reading and
Estimation
Foundations, Forms, and
Concrete
Total
Thank You!
Questions?
www.nocti.org