Proceedings of the 2nd International Conference of Teaching and Learning (ICTL 2009)
INTI University College, Malaysia
ASSESSING THE ATTAINMENT OF COURSE OUTCOMES
(CO) FOR AN ENGINEERING COURSE
Izham Zainal Abidin1, Adzly Anuar2 and Norshah Hafeez Shuaib3
Universiti Tenaga Nasional, Malaysia ([email protected]; [email protected]; [email protected])
ABSTRACT
One of the key aspects in Outcome-Based Education (OBE) is the assessment of the learning outcomes. The
College of Engineering in UNITEN has adopted OBE for all its bachelor degree programmes since the past 2
years. At the early stage of OBE implementation, the specific learning outcomes or also known as Course
Outcomes (CO) for each course were drawn up based on the Programme Outcome (PO) and other requirements.
COs are the attributes, that the student is expected to have or obtained at the time he or she completing the
course. Each course under the bachelor programme has its own set of COs. At the end of each course, the COs
need to be assessed and evaluated to check whether it has been attained or not. A method to evaluate the
achievement or attainment of the COs has been developed. This paper describes the background of the method,
how the method is used, and the results produced. The method utilises data obtained from students’ marks in
exams, test, project and other formal assessments. A computerised system has been developed based on this
method to expedite the analysis process. The findings are then used for continuous quality improvement.
KEYWORDS
Attainment, OBE, Learning outcomes, Continuous quality improvement, Assessment
INTRODUCTION
Over the past two years, the College of Engineering at the Universiti Tenaga Nasional
(UNITEN) has adopted OBE approach in its teaching and learning process. OBE
implementation is nowadays compulsory in attaining accreditation from Engineering
Accreditation Council (EAC), Malaysia [1]. EAC has adopted the OBE principles from
American Accreditation Board of Engineering and Technology (ABET), where the learning
outcomes for each course need to be measured and used for continual quality improvement
[2]. OBE focuses on outcomes that are identified and measured. These outcomes attributes,
normally in form of knowledge, skill or attitude, which prepare the graduates for their
professional practice [3]. The outcomes are normally looked at 3 different levels, at course
level (Course Outcomes), at programme level (Programme Outcomes), and at professional
level (Programme Objectives).
This paper describes a method that is used to analyse or evaluate the attainment of specific
learning outcomes which are the Course Outcomes (CO) for a course. CO is the attribute that
the students are expected to have after completing the course. The evaluation of whether CO
is attained is essential in determining the student achieving in a particular course. The result
of CO attainment will also be used to evaluate the attainment of Programme Outcomes (PO).
The outcome of analysis will be used to improve the teaching and learning experience in the
particular course.
1
BACKGROUND
All courses under the bachelor degree programme would have their own course outcomes or
also commonly known as CO. These COs are produced based on the requirement of the
programme outcomes (PO). Each CO will be mapped to PO (the CO-PO) matrix. The PO
will be then mapped to the programme educational objectives (PEO). Figure 1 shows an
example of relationship between CO, PO and PEO.
Figure 1: An example of the relationship between CO, PO and PEO
Once the outcomes for a course have been identified, the delivery and assessment methods
need to be designed in such a way to achieve the stated outcome. The delivery is important to
ensure the student able to acquire the knowledge or skill required. Assessment is also
important to assess whether the student or learner has attained what is expected out of them.
The result from these will be used for continuous quality improvement (CQI). Suitable
assessment method needs to be chosen, depending on the expected course outcome and the
delivery method [4,2].
In order to acquire the correct feedback for CQI, suitable assessment method is required. The
type of assessment methods would also be dependent upon the expected course outcome and
the delivery method. A summary on the role of assessment is illustrated in Figure 2.
Figure 2: Block diagram showing the relationship between assessment, delivery and the
outcome in OBE
2
In general, assessment method is a huge topic by itself and can be divided into two major
groups – formative assessment and summative assessment [5, 6]. It is often difficult to place
a certain assessment type into either formative or summative. Formative assessment is said to
be part of the delivery or instructional process, where it is used to gather information and
adjust the teaching and learning in ‘real-time’ [6]. Summative assessment is more common
approach, where the students are given a standardized test or examination at a certain period
of time, such as mid-semester test or final exam, which is also known as formal assessment.
This type of assessment is used to test what the student knows at that particular point of time
and whether he or she has met the course goal or the learning outcome.
In this paper, focus is given to using formal assessment data to evaluate the attainment of
course outcomes. Formal assessment generally refers to the standardised assessment that is
used in evaluating the student for their grade. At present, the common method used in
assessing the student in the department includes final exam (which is normally the highest
percentage), mid semester test, quizzes, assignments, mini projects and lab reports (for labbased courses). The data used for evaluating CO attainment is obtained from the students’
mark from these assessments.
CO-ATTAINMENT METHOD
Generally, the approach in evaluating the attainment of CO is using existing data from
students’ marks, for example from the test results, final exam, quizzes and lab report. These
assessments are referred as formal assessment. The method is also sometimes known as
course embedded measurement. This method is chosen because of the information is readily
available and it is common for most courses. In general, assessment methods used are
grouped into 6 categories: (1) Final exam (2) Tests (3) Quizzes (4) Assignments (5) Project
(6) Special. Each of these categories contributes a certain portion of the marks into some of
the COs. The first 5 categories are self-explanatory, while “Special” is used when a certain
assessment could not be placed in any other categories, such as oral presentation.
Assessment-CO matrix is produced for each individual course based on these 6 assessment
categories. This matrix shows the weightage distribution of the percentage of marks
distribution for each specified CO. Table 1 shows a general form of assessment-CO matrix.
The number in the matrix shows the amount, in term of percentage, contributes for each CO.
Table 1: An example of Assessment-CO matrix
Assessment Type
Final Exam
Test
Quizzes
Assignments
Project
Special
Course Outcomes (CO)
CO1 CO2 CO3 CO4 CO5 CO6 CO7 CO8 CO9 CO10
20% 40% 40%
30% 30% 40%
30% 30% 20% 20%
10% 20% 50% 20%
20% 20% 20% 20%
20%
3
In the example above, the Final Exam contributes to three outcomes (CO2, CO3 and CO4)
with different percentages. Other types of assessment contribute differently to the outcomes
as the number or percentage shown in the matrix.
Each CO will have its on ‘mark’ based on the percentage given. The ‘CO-mark’ is calculated
based on the percentage of marks distribution for each assessments group. Using the above
example:
MarksCO1 = 0% of Final exam (FE) + 30% of Tests (T) + 30% of Quizzes (Q)
+ 10% of Assignments (A) + 0% of Project (P) + 0% of Special (S)
Since it is quite common that each assessment group has different overall percentage, there is
a need to factor in the different weight into the calculation. Table 2 shows an example of
overall percentage distribution.
Table 2: Example of overall percentage distribution
Assessment Final
exam
Overall
50%
percentage
Test Quizzes
15%
5%
Assignment Project
10%
20%
Special
-
TOTAL
100%
Based on the overall percentage distribution, the CO-mark needs to be re-calculated so that
the score is normalised accordingly. Using example in Table 2, the new mark for CO1will be:
MarksCO1 = (0.3T x 0.15) + (0.3Q x 0.05) + (0.1A x 0.1)
Similarly, for CO2:
MarksCO2 = (0.2FE x 0.5) + (0.3T x 0.15) + (0.3Q x 0.05) + (0.2A x 0.1)
Where
FE is the student’s Final Exam mark
T is the student’s Test mark
Q is the student’s Quizzes mark
A is the student’s Assignment mark
The students’ marks are tabulated according to the assessment groups. An example of the
mark is shown in Table 3. These marks are then used to calculate the CO-marks.
Table 3: Example of student marks according to each assessment group
Student 1
Student 2
Student 3
Student 4
Student 5
Student 6
Final
exam
77.5
84
67.5
68.5
92
74.5
Test(s)
Quizzes
Assignments
Project/Report
Special*
60.5
92.75
79.75
56.25
72
69
4.83333
9.66667
7.16667
5
9.83333
7.66667
9.75
10
9.875
9.75
10
9.875
12.5
13
12.5
14.5
13.5
12.5
-
4
After the calculation, the new CO-mark for each outcome for each student is tabulated in a
new table, as shown in Table 4. In this table, the CO marks for individual student according
to each CO is presented. The user can set a threshold value for CO attainment criteria, for
example 70%. Using the data in Table 4 as an example, it is observed that Student1 has
attained the entire COs except CO1. From this data, the attainment for each CO for this
course can be analysed by counting the number of students attaining that particular CO, or by
calculating the average score.
Table 4: Example of CO-mark result
Student
Student 1
Student 2
Student 3
Student 4
Student 5
Student 6
CO1 CO2 CO3 CO4 CO5 CO6 CO7 CO8
63
73
77 78.77 83.33 83.33 83.33 NR
95
89
89 86.05 86.67 86.67 86.67 NR
81
74
75 72.31 83.33 83.33 83.33 NR
61
67
70 74.14 96.67 96.67 96.67 NR
82
88
90 92.53
90
90
90
NR
75
76
77 77.69 83.33 83.33 83.33 NR
CO9 CO10
NR 83.33
NR 86.67
NR 83.33
NR 96.67
NR
90
NR 83.33
An example of CO attainment result for a course is shown in Table 5.
Table 5: Example of CO attainment result
Course Outcomes
Attainment result
Score
CO1 CO2 CO3 CO4 CO5 CO6 CO7 CO8 CO9 CO10
NO YES YES YES YES YES YES YES YES YES
48% 70% 74% 83% 97% 97% 97% NR NR 97%
CO Attainment MS Excel Sheet
A system was developed using Microsoft Excel software to assist the lecturers in determining
the CO attainment. The lecturer needs to key in the CO-PO matrix (Figure 3) and the data in
the Assessment-CO matrix (Figure 4) accordingly. Then, the students’ marks are keyed in.
Once the marks are keyed in, the system will automatically calculate the CO attainment. The
result is shown as graph form (Figure 5).
Figure 3: A screenshot showing the CO-PO matrix for MEMB263
5
Figure 4: A screenshot showing the Assessment-CO matrix for MEMB263
Figure 4 shows an example of the CO attainment graph. This course only has 7 CO, therefore
CO8 to CO10 are not displayed. The graph shows the percentage of the student attaining the
CO (percentage of YES). In this example, the student needs to obtain the score of 70% or
more in order to attain the CO. The marks from 60 students (one section) are analysed to see
their scoring. A rule is set where at least 40% of the students (i.e. 24 persons) need to obtain
the score of 70% or more in order for this course’s CO is attained. These rules may be
changed accordingly, depending on the target of the college.
Figure 5: An example of CO attainment result shown in graph form.
The College has been using this system for the past 3 semesters. It has been observed that the
system is very useful in helping the lecturers to carry out the CO attainment analysis. The
result is automatically produced and used to identify any outcome that would require further
attention and improvement.
Based on the result, action plan can be identified and implemented in the coming semester.
The whole process will become the course-level attainment-loop where the result is being fed
back into the system for improvement. The result can be also used in programme-level CQI
loop where the attainment of the programme outcomes (PO) can be measured and analysed.
6
CONCLUSIONS
A method to find CO attainment has been developed and implemented on a computerised
system using Microsoft Excel software. This direct measurement method provides a
structured way to analyse the attainment outcomes for each course. The students’ marks from
formal assessment such as final exam, test and project are used to calculate the CO-mark for
that particular course. From this result, the attainment of each course outcome for the course
can be further reviewed and analysed. Action plan to improve any weakness can be identified
and implemented in the following semester.
The system has been found to be very helpful in the analysis process. Using this system, the
lecturer could key in the relevant data and obtain the CO attainment for his or her course. The
analysis and result will be automatically produced. This system is continuously being
improved and helpful features are being added from time to time.
REFERENCES
1.
Engineering Accreditation Council (EAC), Engineering Programme Accreditation
Manual, Board of Engineers Malaysia (BEM), 2007.
2.
M.S. Jaafar, N. K. Nordin, R. Wagiran, A. Aziz, M.J.M.M. Noor, M.R. Osman, J.
Noorzaei and F.N.A. Abdulaziz, “Assessment Strategy for an Outcome Based
Education”, International Conference on Engineering Education, July 2008.
3.
H. Basri1, A. B. Che Man, W. H. Wan Badaruzzaman and M. J. M. Nor, “Malaysia And
The Washington Accord: What It Takes For Full Membership”, International Journal of
Engineering and Technology, Vol. 1, No. 1, 2004, pp. 64 – 73.
4.
Rozeha A Rashid, Razimah Abdullah, Azami Zaharim, Hamzah Ahmad Ghulman and
Mohd Saidfudin Masodi, “Engineering Students Performance Evaluation of Generric
Skills Measurement: ESPEGS Model”, 5th WSEAS / IASME International Conference
on Engineering Education (EE'08), July 22-24, 2008
5.
Active Learning for Critical Thinkers, University of Texas at Arlington. Retrieved May
19, 2009, from http://activelearning.uta.edu/FacStaff/formsum.htm
6.
Formative and Summative Assessment in the Classroom. National Middle School
Association, USA. Retrived May 20, 2009, from http://www.nmsa.org/Publications/
WebExclusive/Assessment/tabid/1120 /Default.aspx
7