Criterion 2. Program Educational Objectives

MÜHENDİSLİK FAKÜLTESİ
2015 ABET DEĞERLENDİRME SÜRECİ
ÖNEMLİ KONULAR
ABET Symposium 2014
Fundamentals of Assessment Workshop
Advanced Assessment Workshop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
12 Mayıs 2014
TOPLANTININ AMACI
2015 ABET Next General Review
akreditasyon sürecine hazırlanırken
önemli bazı konuların hatırlatılması
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ODTÜ ve ABET
2009
2004
1994
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
2001
1996
ODTÜ ve ABET
• Tüm Bölümlerimiz en az iki kez ABET
tarafından ziyaret edildi:
 3 kez: CHE, MINE, CE, EE, ME, METE
 2 kez: AEE, CENG, ENVE, FDE, GEOE, IE,
PETE
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
TOPLANTININ AMACI
• Önceki değerlendirme süreçlerinde ifade edilen
eksiklikler/iyileştirme önerileri - hatırlatma
• ABET kapsamında yapılan değişiklikler
– Bölümlerimizle daha önce yazılı olarak paylaşılanlar
– Fakültemiz tarafından geniş katılım sağlanan Nisan
2014 ABET sempozyum ve çalıştaylarında
vurgulanan konular
• ABET akreditasyonu konusunda tecrübeli bir
kurumdan olası artan beklentilerin karşılanması,
sistematik olarak uygulanan, yerleşmiş bir sistemin
gösterilmesi gerekliliği
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
BİLGİLENDİRMELER - CD
 3-4 Nisan 2014 ABET Symposium
 5 / 6 Nisan 2014
Fundamentals of Program
Assessment Workshop
 6 Nisan 2014
Advanced Program
Assessment Workshop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ÖNEMLİ DOKÜMANLAR
Most recent (2014-2015) versions of :
• Self Study Questionnaire (SSQ)
• Accreditation Policy and Procedure
Manual (APPM)
• Criteria for Accrediting Engineering
Programs (Criteria)
Shortcoming can be written due to lack of issues
from SSQ or APPM.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ABET TAKVİMİ
 ABET ziyareti öncesi
 Başvuru - Dekanlık: 15 Ocak 2015
 Bölümlerin raporlarını Dekanlık’a sunmaları: 26 Ocak 2015
 Raporların ön değerlendirmeden geçmesi: Şubat 2015
 Raporların ön değerlendirme sonrası tamamlanması: 3 Nisan 2015
 Fakülte raporunun hazırlanması: Nisan 2015
 Fakülte raporu ile birlikte tüm raporların tamamlanması: 8 Mayıs 2015
 Raporların basılması: Mayıs 2015
 Raporların ABET’e gönderilmesi: 8 Haziran 2015
 ABET ziyareti: Ekim 2015 (Pazar – Salı)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ABET TAKVİMİ
 ABET ziyareti sonrası (Due Response Process)
 Exit meeting – Program Audit Forms: Ekim 2015
 7-day response by METU (errors of fact only): Ekim 2015
 Draft Statement from ABET: Ocak-Şubat 2016
 30-Day Due Response by METU: Şubat – Mart 2016
 ABET Annual EAC Committee Meeting: Temmuz 2016
 Final Statement from ABET: Ağustos 2016
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From APPM
Shortcoming
What it means
Deficiency
criterion, policy, or procedure is not satisfied. Therefore, the
program is not in compliance with the criterion, policy, or
procedure
Weakness
a program lacks the strength of compliance with a criterion,
policy, or procedure to ensure that the quality of the program
will not be compromised. Therefore, remedial action is
required to strengthen compliance with the criterion, policy, or
procedure prior to the next review
Concern
a program currently satisfies a criterion, policy, or procedure;
however, the potential exists for the situation to change such
that the criterion, policy, or procedure may not be satisfied
Observation
a comment or suggestion that does not relate directly to the
current accreditation action but is offered to assist the
institution in its continuing efforts to improve its programs
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From APPM
Accreditation Action
What it means
Typical Duration
Next General Review
No Deficiency, No Weakness
6 yrs
Interim Report
One or more Weaknesses
2 yrs
After an IR
2 or 4 yrs
One or more Weaknesses –
re-visit required
2 yrs
After an IV
2 or 4 yrs
One or more Deficiencies
2 yrs
After a SC
2 or 4 yrs
One or more Deficiencies –
re-visit required
2 yrs
Report Extended
Interim Visit
Visit Extended
Show Cause Report
Show Cause Extended
Show Cause Visit
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
RECENT ABET CHANGES
• Criterion 4 no longer requires that the
extent to which graduates attain Program
Educational Objectives be measured
• Criterion 2 still requires that the Program
Educational Objectives be periodically
reviewed to ensure that they are
consistent with the institutional mission
and constituency needs
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
Criterion 4 Change
• Be careful about the ADDED WORDS to the
criterion
– Regularly
– Appropriate
– Systematically utilized
• You have to document your evaluations &
decisions on assessment results
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
MAIN ISSUES
• A systematically utilized, well documented
(evidence-based) program assessment process
– Every 6 yrs is not a good cycle
– Assessing every course every semester is not
sustainable or recommended
• A good understanding of student strengths &
weaknesses
– General % success of Student Outcome attainment not
enough
– Show the level of attainment & how it changes in time
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
MAIN ISSUES
• Focus on how the assessment process
help improve the program
– Show how assessment help improve program
(close the loops – continuous improvement)
– If possible, for more than one-cycle since
2009 ABET visit
– Link them to improvement/change in Student
Outcome attainment
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
MAIN ISSUES
• Now, much more emphasis on
assessment & attainment of Student
Outcomes
• After all the assessment you are doing:
What are your students’ strengths and
weaknesses related to this SO?
Do the students know what outcomes are
expected of them?
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
MAIN ISSUES
 Although not required, ABET strongly
recommends using PERFORMANCE
INDICATORS (PIs) for assessment of Student
Outcomes
– PIs vs. SOs
– Courses vs. PIs
Instead of our typical
– Courses vs. SOs
– PIs enable a clear definition of what each SO
mean for the program. Otherwise there is no
common view among faculty.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
MAIN ISSUES
Cont. improvement ≠ Cont. Assessment
 Choose a set of indicators for SOs
 Select courses that provide info for indicators
 Assess courses in a 2- or 3-yr cycle
 You DO NOT have to set the same target of
achievement for all SOs.
 Technical SOs can be more important
 If you have numerous info sources for one SO, but
much less for another, then a lower target can be
set for the latter
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SO Attainment via Performance
Indicators
MÜHENDİSLİK FAKÜLTESİ
ABET
From2014
Fri
12 Mayıs
9:15, Diana Briedis, Closing the Loop
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
WHAT ELSE IS IMPORTANT
APPM
• ODTÜ – KKK programları da olan bölümlerin
Ankara kampüsündeki programın ABET
tarafından akredite olduğu ve diğerinin olmadığı
konusunda web sayfası ve basılı/elektronik
katalog bilgilerinde farkın çok açık ve net olması
gerekiyor.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
WHAT ELSE IS IMPORTANT
APPM
• ODTÜ – Ankara kampus kataloğunda PEO, SO
bilgileri basılı/elektronik katalog ve web sayfası
ile uyumlu, en güncel hali olmalı
• Tüm Bölümlerin en güncel APPM’e göre
aşağıdaki şekilde yazması gerekiyor:
“The *** engineering undergraduate program was
accredited by the Engineering Accreditation
Commission of ABET, http://www.abet.org.”
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
WHAT ELSE IS IMPORTANT
Program Criteria
• Program Criteria dokumanında birden fazla
bölümümüz programı için önemli olabilecek bazı
değişiklikler oldu. Bunların müfredat ile
karşılaştırmalı olarak incelenmesi çok kritik.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
Facility
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Team coming in 2015 may cite a shortcoming
for which the team in 2009 did not state as a
shortcoming, even though that process did
not change for the institution between 2009 to
2015.
• This is completely acceptable by ABET
• ABET says this is why an evaluator only
evaluates a program once!
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ESPECIALLY GOOD TALKS
ABET 2014 SYMPOSIUM
• ACCREDITATION TRACK – on CD
Thu 15:15 “Preparing the Self-Study Report
for Engineering”, Jeff Fergus
Fri 9:15 “Demonstrating Compliance with
Criterion 4: Closing the Loop”, Diana Briedis
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ÖZETLE
• Artik 3./4.kez denetime gelindigi icin
beklentiler daha yüksek olacak
• PEO attainment requirement ortadan
kalktığı için SO attainment önem kazanmış
durumda
• Ögrencilerin kazanımlarında zayıf ve güçlü
yönlerin net olarak belirlenmiş olması
bekleniyor (bkz. SO için indikatör kullanımı)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ÖZETLE
• Sistematik ve kanıt-bazlı bir değerlendirme
sisteminin yerleşmiş ve kullanılıyor olması
bekleniyor. Program iyileştirmelerinin
büyük oranda buna bağlı olması
bekleniyor.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Please read:
– The previous Self Study Report prepared by
your Department in 2009
– The previous ABET Evaluation Report
prepared by the team after the site visit
• Please check out for valuable info:
http://www.abet.org/accreditation
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
TEŞEKKÜRLER
YOU DON’T HAVE TO BE BAD TO GET BETTER!
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
The Criteria
•
•
•
•
•
•
•
•
•
Criterion 1: Students
Criterion 2: Program Educational Objectives
Criterion 3: Student Outcomes
Criterion 4: Continuous Improvement
Criterion 5: Curriculum
Criterion 6: Faculty
Criterion 7:Facilities
Criterion 8: Support
Criterion 9: Program Criteria
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
SON HALİ (2014-2015 Criteria for Accrediting Engineering Programs)
Criterion 2. Program Educational Objectives
The program must have published program educational objectives that are
consistent with the mission of the institution, the needs of the program’s various
constituencies, and these criteria. There must be a documented,
systematically utilized, and effective process, involving program
constituencies, for the periodic review of these program educational
objectives that ensures they remain consistent with the institutional
mission, the program’s constituents’ needs, and these criteria.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
ÖNCEKİ HALİ (2011-2012 Criteria for Accrediting Engineering Programs)
Criterion 2. Program Educational Objectives
The program must have published program educational objectives that are
consistent with the mission of the institution, the needs of the program’s various
constituencies, and these criteria. There must be a documented and effective
process, involving program constituencies, for the periodic review and
revision of these program educational objectives.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
YAPILAN DEĞİŞİKLİK HAKKINDA AÇIKLAMA
The change to Criterion 2 removes the requirement for a program to have a
process for the "review and revision" of its program educational
objectives (PEOs). The new language requires a documented process that is
systematically utilized and effective, involving the program's constituents, for
periodic review of the PEOs so that they remain consistent with the institution's
mission, the constituents' needs, and the criteria.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
SON HALİ (2014-2015 Criteria for Accrediting Engineering Programs)
Criterion 4. Continuous Improvement
The program must regularly use appropriate, documented processes for
assessing and evaluating the extent to which the student outcomes are
being attained. The results of these evaluations must be systematically utilized
as input for the continuous improvement of the program. Other available
information may also be used to assist in the continuous improvement of the
program.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
ÖNCEKİ HALİ (2011-2012 Criteria for Accrediting Engineering Programs)
Criterion 4. Continuous Improvement
The program must regularly use appropriate, documented processes for
assessing and evaluating the extent to which both the program educational
objectives and the student outcomes are being attained. The results of
these evaluations must be systematically utilized as input for the continuous
improvement of the program. Other available information may also be used to
assist in the continuous improvement of the program.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
YAPILAN DEĞİŞİKLİK HAKKINDA AÇIKLAMA
The change to Criterion 4 removes the requirement for a program to
demonstrate graduate attainment of program educational objectives. This
change removes the stringent requirement for assessment of program
educational objectives as is required for a program's student outcomes.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
SON HALİ (2014-2015 Criteria for Accrediting Engineering Programs)
Criterion 6. Faculty
The program must demonstrate that the faculty members are of sufficient
number and have the competencies to cover all of the curricular areas of the
program. There must be sufficient faculty to accommodate adequate levels of
student-faculty interaction, student advising and counseling, university service
activities, professional development, and interactions with industrial and
professional practitioners, as well as employers of students.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
ÖNCEKİ HALİ (2013-2014 Criteria for Accrediting Engineering Programs)
Criterion 6. Faculty
The faculty must be of sufficient number and must have the
competencies to cover all of the curricular areas of the program. There must
be sufficient faculty to accommodate adequate levels of student-faculty
interaction, student advising and counseling, university service activities,
professional development, and interactions with industrial and professional
practitioners, as well as employers of students.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
YAPILAN DEĞİŞİKLİK
Criterion 6. Faculty: The program must demonstrate that the faculty
membersmust beare of sufficient number and theymust have the
competencies to cover all of the curricular areas of the program.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
YENİLİKLER
6 Kasım 2013 - ABET tarafından yayınlanan ALERT
1. Each ABET-accredited program must publicly state the
program educational objectives and student outcomes
2. Each ABET-accredited program must publicly post annual
student enrollment and graduation data per program
*** 2014 YILI BAŞINDAN İTİBAREN UYGULANMAK ÜZERE!
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
DEĞİŞİKLİKLER
 Program Criteria Changes
 Bölümler kendi lisans programlarına etki edebilecek
değişikliklere muhakkak bakmalı.
 Revisions to the following Program Criteria will be in effect for the
2014-15 review cycle.
– Electrical, Computer, Communications Engineering
– Environmental Engineering
– Mechanical Engineering
 Revisions to the following Program Criteria are offered on first
reading for review and comment. See the Proposed Changes
Section of the EAC Criteria.
– Electrical, Computer, Communications Engineering
– Materials (1), Metallurgical (2), Ceramics (3) Engineering
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
Facility
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Jeff Fergus
Accreditation Track, Thu 15:15
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
Criterion 4 – Closing the loop
• Most important part of the Self-Study
Report
• Greatest number of shortcomings listed for
this Criterion!
• You HAVE TO show that some
improvements (especially major program
improvements) are tied to the assessment
data & evaluation of that data!
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
Criterion 4 from Program Criteria
• Criterion 4. Continuous Improvement
• The program must regularly use appropriate,
documented processes for assessing and evaluating the
extent to which the student outcomes are being attained.
The results of these evaluations must be
systematically utilized as input for the continuous
improvement of the program. Other available
information may also be used to assist in the continuous
improvement of the program.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
Criterion 4 from SSQ
• Continuous Improvement
• Describe how the results of evaluation processes for the
student outcomes and any other available information
have been systematically used as input in the
continuous improvement of the program. Describe
the results of any changes (whether or not effective) in
those cases where re-assessment of the results has
been completed. Indicate any significant future program
improvement plans based upon recent evaluations.
Provide a brief rationale for each of these planned
changes.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
Criterion 4 Change
• Emphasis on new requirement of Cr 4
– Alumni survey, Employer survey  no longer
mandatory
– BUT These are ‘program constituencies’ therefore we
need their input at some stage
– Industrial advisory board may need to meet more
frequently (e.g. 2 yrs)  due to Criterion 2 changes
Instead of ATTAINMENT now
APPROPRIATENESS
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Thu 15:15, Karen Tarnoff, Preparing Effective Assessment Documentation
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Thu 15:15, Karen Tarnoff, Preparing Effective Assessment Documentation
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Thu 15:15, Karen Tarnoff, Preparing Effective Assessment Documentation
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
After all the assessment you are doing in your
program, can you answer the following question at
the end of the day:
WHAT ARE YOUR STUDENTS’
STRENGTHS & WEAKNESSES
RELATED TO THIS STUDENT OUTCOME?
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
1. We are not doing COURSE assessment,
but PROGRAM assessment!
2. Although not REQUIRED, ABET strongly
recommends using PERFORMANCE
INDICATORS (PIs) for assessment of Student
Outcomes
– PIs vs. SOs
– Courses vs. PIs
Instead of our typical
– Courses vs. SOs
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Thu 15:15, Kathy Faggiani, Assessment and the Big Bang
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fundamentals of Program Assessment Workshop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fundamentals of Program Assessment Workshop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Fred DePiero, Sustainable Assessment Processes via Centralized Efforts
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
Assessment Methods
From Fundamentals of Program Assessment Workshop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PI examples
• Writing measurable outcomes
• PEO  SO  PI
– Observable, measurable
– Shouldbe something you see demonstrated in
student work
– Sample Pıs
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SO Attainment via Performance
Indicators
MÜHENDİSLİK FAKÜLTESİ
ABET
From2014
Fri
12 Mayıs
9:15, Diana Briedis, Closing the Loop
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From Fri 9:15, Diana Briedis, Closing the Loop
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
3. THEN, they recommend using rubrics for
grading PIs so that across different
courses, even grading can be achieved
(inter-rater reliability)
4. SO assessment is best done in 3-yr
cycles
1. Collect
2. Evaluate
3. Implement
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS - Example
• Course assessment in the context of a sustainable
improvement process (Zia Yamayee) suggested:
– Phase I: from last final statement to year of self study (4
years)
– Phase II: year of self study (1 academic year)
– Phase III: on site review (1 year)
• In phase I:
–
–
–
–
Periodic review of PEOs (every 3 years)
Assessment & evaluation of SOs (every other year)
Identify & implement improvements (annual)
Document (annual)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
5. Overall course grades for assessment is
NOT a good idea
– homeworks, mid-term grades, etc. are OK
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
6. Do not try to collect data from ALL courses for
SO assessment. Choose your courses as to
how much each contribute to a PI or SO.
–
–
–
In Advanced Assessment workshop, they showed a
mapping for courses vs. SOs & they normalized on
courses instead of SOs
A course is not recommended to be used directly for
the assessment of the level of achievement of a SO.
(Use PIs which cover a number of courses)
Do not use Pass/Fail type of scales in assessing
attainment of SOs or PIs in a course work
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
7. During SO assessment you can have time depth
–
–
–
focusing on measurement of juniors and seniors more
than freshmen and sophomores. (see ppt by Elias
Nassar: Building and assessment and CI system)
Show the level of attainment throughout the curriculum
Select benchmark courses for assessment of SOs. Do
not overdo by selecting every related course every
semester
Note: Initial assessment of SO for communication skill may not be made on ENG
101 courses, but instead Introduction to *** Engineering courses can be used for
this purpose!
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
8. Continuous improvement does not mean
continuous assessment!
– A three year assessment cycle is acceptable &
recommended because
•
•
it is more sustainable
Makes it much easier to involve faculty
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
9. You can give different level of attainment
targets for different SOs. They do not have to
be equal!
–
–
–
–
Technical SOs can understandibly be more
important for the program
For ex. a, b can have 90% as a target
But for ex. k can have 60% as a target
This is because technical ones can be more
important for you, or you may have more chances in
your program to evaluate/assess them.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
10.Program evaluator will ask ‘So, what are the
strengths & weaknesses of your students
related to this student outcome?’
– You should have an answer for that and
using performance indicators (but not
necessarily courses) allows for that
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
11.Avoid a «stealth» assessment process.
Students should know they are being
assessed & they should be
knowledgeable about SOs & the level of
performance expected of them.
 Students are «drinking from a fire-hose»
therefore they learn better if they know what’s
expected of them – the learning objectives
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
12.Typically largest number of shortcomings
are given from Criterion 4!
MÜHENDİSLİK
FAKÜLTESİ
From Fri 9:15, Diana Briedis, Closing the Loop
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
13. Make sure you have evidence of
assessment & evaluation
Make sure they are present in the folders
kept in the display area
Try to show the course materials for the most
recent offering. You do not need to archive all
the past material (Joe Turner, Thu 9:15)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ASSESSMENT PROCESS
14. It is advisable to have an even number of
scale for SO assessment :
• Bloom’s taxonomy
• EMAU
• Four column template
For example:
1.
2.
3.
4.
Exceeds standard (Excellent/exemplary)
Meets standard (Good/Satisfactory)
Progressing to standard (Fair/Developing)
Below standard (Poor/Unsatisfactory)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
FACULTY
• All faculty needs to be involved at some
stage
• Evaluator will interview faculty and get a
feel of the assessment culture of the
program
– Typically Monday afternoon of the site visit
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ABOUT ABET 2014 Symposium
• A CD is prepared which includes the
presentations of all talks in the ABET 2014
Symposium in Pittsburgh, PA, USA (3-4 April,
2014)
• CD also contains workbooks of
– Fundamentals of Program Assessment Workshop
– Advanced Assessment Workshop
– Other useful documents
• One CD is sent to each Department
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ESPECIALLY GOOD TALKS
AT THE SYPOSIUM
• ACCREDITATION TRACK
– Thu 9:15 "Demonstrating Compliance with
Criterion 4: Assessment and Evaluation", Joe
Turner
– Thu 15:15 “Preparing the Self-Study Report
for Engineering”, Jeff Fergus
– Fri 9:15 “Demonstrating Compliance with
Criterion 4: Closing the Loop”, Diana Briedis
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ESPECIALLY GOOD TALKS
AT THE SYPOSIUM
• PROGRAM ASSESSMENT TRACK
– Thu 9:15 ”How to Embed Measurable
Assessment Instruments into a CE Capstone
Design Course", Ivonne Santiago (invited)
– Thu 15:15 “Common Mistakes During the
Assessment Process”, Gloria Rogers
– Fri 9:15 “Sustainable Assessment Processes
via Centralized Efforts”, Fred DePiero (invited)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ESPECIALLY GOOD TALKS
AT THE SYMPOSIUM
• PROGRAM ASSESSMENT TRACK
– Thu 15:15 "Preparing Effective Assessment
Documentation", Karen Tarnoff
– Fri 8:00 "Building an Assessment and
Continuous Improvement System: How to help
faculty with the initial steps", Elias Nassar
– Thu 14:00 "Course Assessment in the Context
of a Sustainale Continuous Improvement
Process", Zia Yamayee
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ESPECIALLY GOOD TALKS
AT THE SYMPOSIUM
• PROGRAM ASSESSMENT TRACK
– Fri 9:15 "Experiences Obtaining Data on
Student Outcomes from Capstone Design
Projects", John Schlup
• ACCREDITATION TRACK
– Fri 14:00 “The Engineering Body of
Knowledge”, Craig Musselman
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
GOOD TO READ
• A self-study template was prepared at
CalPoly that includes good examples of
presentation of results, etc.
•
Self-Study Template (Word doc)
Detailed instructions for writing the self-study; may serve
as outline of self-study report
http://www.academicprograms.calpoly.edu/accred_progrev/
ALSO
sites.google.com/site/selfstudycalpolyslo/
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
GOOD TO READ
ABET Program Evaluator (PEV) Training
Documents
• http://www.abet.org/pev-candidatetraining/
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Course & assessment material display area is
VERY IMPORTANT
• The evaluator will study these on the Sunday
afternoon (13:30-onwards)
• It may be a good idea to arrange the display
materials according to Student Outcomes
(e.g.one folder per SO)
• Make it easy for the evaluator to find information
(both in report & display area)
• Have evidence from the last cycle at a minimum
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
Organize materials to allow an “audit” process
• Course materials should demonstrate any data
collected from student performance in the
courses
• Evaluator should be able to trace:
1. From data collection to evaluation
2. To use of evaluation results in determining whether
there is potential need for improvement
3. To any improvement action that was implemented
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
For currently accredited programs:
– Assessment data summaries
– Evaluations
– Use in identifying any potential improvements
since last ABET review should be in display area
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Although assessment is done for
departmental courses, if during the visit the
evaluator asks, then evidence should be
made available from other departments
– Physics, Math, English, etc.
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Team coming in 2015 may cite a
shortcoming for which the team in 2009
did not state as a shortcoming, even
though that process did not change for the
institution between 2009 to 2015.
• This is completely acceptable by ABET
• That is why an evaluator only evaluates a
program once!
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Must read:
– The previous Self Study Report prepared by
your Department in 2009
– The previous ABET Evaluation Report
prepared by the team after the Site Visit
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
PREPARATION FOR SITE VISIT
• Early communication of the evaluator and
Team Chair with the Dean or Dept Chair
can allow the program to actually revise
some things
• A shortcoming may even be eliminated by
conducting such a process
(But this depends on the preference of the
Team Chair)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SUMMARY TIMELINE
Due Process (Post-Visit Activities)
• Exit Meeting: PAF (Program Audit Form)
given to Dean
– Specifies shortcomings and explanation
• Institution’s 7-Day Response
– Errors of fact only
• Draft Statement of Findings
–Prepared by team, edited at two levels, then
sent to the institution (early spring)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SUMMARY TIMELINE
Due Process (Post-Visit Activities)
• Institution’s Due Process Response
– Due within 30 days of receipt of draft
statement
– Programs are encouraged to solve problems
quickly
– Describe and document changes since the
visit
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SUMMARY TIMELINE
Due Process (Post-Visit Activities)
• Final Statement
– Incorporates due process response
– Final Statement serves as the basis for the
commission (EAC for engineering programs)
action on accreditation at July meeting
– Final Statement and accreditation letter sent
by ABET to the institution (by August 31st )
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
From the talk
of Jeff Fergus
(Accreditation
Track,
Thu 15:15)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SUMMARY TIMELINE
Timeline
Process
July
Report submission to ABET
October
Team Visit
Jan-Feb
Draft report from ABET team
~ March
30-day Due Response
July
Accreditation decision
This timeline shows that a number of problematic
issues in a program can actually be eliminated if Team
Chair communicates this info to the Dept between
August/Sept until July of the next year
(special words like Deficiency, Weakness are avoided during
these discussions btw evaluator & Dept Chair)
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
SITE VISIT TIMELINE
Timeline
Process
Saturday
Arrival
Sunday
10:00
Campus arrival
10:00 – 12:30
Visit departments, laboratories,etc.
12:30 – 13:30
Lunch
13:30 – onwards
Work on display materials
Monday
Departmental visit
Tuesday
Closing meeting and departure
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ENGINEERING BODY OF
KNOWLEDGE
• A report prepared by a large group of
engineers (including professionals as well
as academicians) on capabilities of
engineers 30 categories...
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014
ABET Accreditation outside the US
• ABET now accredits 324 programs at 64 institutions in
23 countries outside the US
• About program evaluators:
MÜHENDİSLİK FAKÜLTESİ
ABET
12 Mayıs 2014

Download Report