Policy on the Evaluation of Teaching and Papers Guidelines for the Review, Evaluation and Continuous Improvement of Teaching and Papers LY T N E Teaching Development | Wāhanga Whakapakari R Ako HE D. R T U IN TE C IS ND PDA T N WA U E M VIE ING U C RE BE O F S DDER S O I TH UN ES OC R P Trudy Harris Teaching Development| Wāhanga Whakapakari Ako February 2012 Table of Contents I. Definitions for use with this Document 3 II. Frequently Asked Questions 4 III. Useful Links 4 IV. Introduction 5 V. Evaluation of Teaching and Papers Policy 7 VI. Guidelines for Review, Evaluation and Continuous Improvement of Teaching and Papers 10 1. Summative Tools 10 1.1 The Paper and Teaching Appraisal 10 1.2 Initiating an Appraisal 10 1.2.1 Who? 1.2.2 When? 1.2.3 How? 1.3 What Appraisal Medium? 14 1.4 Conducting the Appraisal 15 1.5 Processing the Appraisal 16 1.6 Interpreting the Appraisal Results 18 1.6.1 The Mean Value 1.6.2 The Distribution Pattern 1.7 Reporting of Appraisal Results 21 1.8 Benchmarking 22 1.9 Appraisal Summaries 22 2. Evaluation for the Enhancement of Teaching 23 3. Formative Tools 24 3.1 Suggestions for Feedback from Students 25 3.1.1 Regular Anonymous Instant Feedback 3.1.2 The Critical Incident Questionnaire 3.1.3 Learning Reflection Exercises 3.1.4 Focus Groups 3.2 Self-evaluation and Reflection 27 3.2.1 Regular Reflection 3.2.2 Keeping a Learning Diary 3.2.3 Compiling and Maintaining a Teaching Portfolio 3.3 Peer Interactions 28 3.4. Submitting Evidence of your Formative Evaluation 29 4. Processes for Continuous Improvement 29 5. Representative Bibliography 30 6. Contact Details 37 LY T N E R THE . R U IN TED C IS ND PDA T N WA U E M VIE ING U C RE BE O F S DDER S O I TH UN ES OC R P APPRAISALS • TDU •2• I. Definitions for use with this Document Appraisal A questionnaire consisting of both paper and teacher appraisal questions. Paper appraisal Refers specifically to the questions associated with paper design and organisation. Teacher appraisal Refers specifically to the questions associated with teaching. Formative evaluation In this context the gathering of qualitative feedback through a variety of different techniques such as peer review, with the aim of enhancing practice. LY T Summative In this context the use of quantitative feedback to N E evaluation assess the quality of paper design and teaching. This E R . H is generally achieved R Tthe useEappraisal Dof the U through University of C Waikato’s formal N questionnaires. S ND I DAT I P medium. Paper-based AppraisalT using paper as a collection A N U appraisal E EW G M Web -based appraisal theIN Internet as a collection medium. U Appraisal VI using E C E B O RE35 sizeF(foolscap) brown or white envelope with Appraisal envelope D O and Confidential Appraisal Forms” written on S DER “Private I S it. H N TAppraisal ES U coverC sheet A form to be completed by teaching staff identifying O how data is to be processed and whether staff wish to see the questionnaires after processing. This form PR is supplied with the appraisal envelope. •3• Appraisal report A PDF report in which each question is summarised as a frequency bar graph. Open question report A PDF file of the open questions responses to an appraisal. Appraisal summary Summary of appraisal results in a few generic sentences for each appraisal. APPRAISALS • TDU ? II. Frequently asked questions: What terminology should we use - ‘Appraisal’ or ‘Evaluation’? Appraisal: Evaluation: The act of measuring the performance of… The use of the information gained through an appraisal to determine the performance of…. In this booklet we use both the terms ‘Appraisal’ and ‘Evaluation’. You will notice that ‘evaluation’ is predominantly used in the Policy while ‘appraisal’ is used in the guidelines. Currently we are moving from a process of appraisal to one of evaluation. ► III. Useful Links LY T N E R THE . R The Appraisal Order Form: U IN TED http://www.waikato.ac.nz/tdu/appraisals/staff/apporder/ C apporder.shtml IS ND PDA T N WA U Item Bank: E M VIE ING http://www.waikato.ac.nz/tdu/pdf/itembank.pdf U E C RE Appraisal B GuidelinesO on Photocopying Questionnaires: F D R http://www.waikato.ac.nz/tdu/pdf/appraisals/guidelines.pdf IS DE S O H SOrder Form: NSummary T Appraisal E U http://www.waikato.ac.nz/tdu/appraisal/summary.shtml OC R P Other links: The following documents can be found at the Teaching Development Unit (TDU) website http://www.waikato.ac.nz/tdu. Evaluation of Teaching and Paper policy: Paper Outline Policy Guidelines for the Review, Evaluation and Continuous Improvement of Teaching and Papers Academic Staff Promotions Criteria Professional Goal Setting Policy (PGS) http://www.waikato.ac.nz/official-info/index/docs/evaluation-ofteaching-and-papers-policy APPRAISALS • TDU •4• IV. Introduction This booklet has been produced as part of the implementation plan of the University of Waikato’s new Policy on the Evaluation of Teaching and Papers. While this booklet contains the new Policy document its main focus is on the guidelines to support the Policy. The new Policy states that over a two year period academic staff members are expected to evaluate their teaching and their papers. The evaluation has to include appraisals, but for development purposes should also utilise one or more of the formative feedback mechanisms outlined in these guidelines. These mechanisms will ensure that paper and teaching quality is of a high standard and that areas that need development are identified. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E M VIE ING U main body ofEthis document falls into two CThe E Summative B and Formative Tools. The O R sections: F D section O on Summative Tools looks at the S DER I S development practice around the use of H T UN EtheS appraisal.of good The appraisal provides the OC University with information for academic R promotion, external audits and internal review. P For this purpose this section provides pointers about ordering appraisals, the type of appraisal to use, the timing of appraisals and also the reporting and benchmarking of results. The second section is dedicated to Formative Tools. A number of tools are described that can aid staff both in obtaining immediate feedback from students, but also for the acquisition of more long term feedback to aid with professional •5• APPRAISALS • TDU development. These tools can provide the rationale for responses given by students to the quantitative appraisal, and can be of great benefit in highlighting good practice and areas of teaching that need development. Contact details for the Teaching Development Unit (TDU) are given at the end of the document. Please feel free to contact the Unit if you have any questions about any of the information included in this document. They would be more than happy to help. LY T N The achievement of consistently good teaching is E R THE . key to the success of our students, fundamental R U to IN TED to the quality of the University and rewarding C each of us as academic staff. I encourage youDto A IS N D familiarise yourself with these T policy guidelines P N WA U and employ them wherever possible. E M VIE ING U C E BE O Professor Doug SuttonR D R OasF Chair of the DeputyS Vice-Chancellor ECommittee I Teaching Quality D H N ESS TJanuaryU2009 OC R P The Policy and Guidelines are in the process of being updated so some of the information in this booklet will change. APPRAISALS • TDU •6• THE UNIVERSITY OF WAIKATO V. Evaluation of Teaching and Papers Policy Responsibility for policy: Chair, Teaching Quality Committee Approving Authority: Academic Board Approved: Next Review: September 2010 LY T N E Scope R chairpersons HE D.of 1. This policy applies to all teaching staffR and theirT U IN TE department (or equivalent). C IS ND PDA T A U Related Documents N E W 2. This policy should M beVread IE inEconjunction NG with the following I U documents: C RE B O Paper F D ROutline Policy, O S E Guidelines for the Review, Evaluation and Continuous I S D H S T UN Improvement E of Teaching and Papers, C Academic Criteria, OProfessionalStaffGoalPromotions R Setting Policy. P Definitions 3. For the purpose of this policy, teaching staff means staff whose conditions of employment are governed by an academic staff employment agreement and whose duties include the design and/ or delivery of paper offerings. •7• APPRAISALS • TDU Purpose 4. Evaluation of Teaching and Papers Policy forms part of the teaching and learning quality assurance framework. Its purpose is to: a. Promote the University’s commitment to the achievement of excellence in teaching and learning by supporting teaching staff to regularly review and evaluate their teaching and papers with a view to ongoing professional development and improvement of teaching; LY T N E R THE . R U IN to TED c. Demonstrate a commitment to continuousC improvement satisfy external academic audit requirements. IS ND PDA T A U N E W Responsibilities M IE EING U 5. Teaching staff are required to: V C RE B O a. Prepare and maintain F up-to-date paper outlines D RaccurateOand S for theI papers forE which they are responsible, in accordance S D H with the Paper Outline T UN ESPolicy; b. Prepare an evaluation OC of all teaching and papers for which they R are responsible P every two calendar years; the evaluation must b. Set out the responsibilities of teaching staff, chairpersons of department (or equivalent) and Deans (or equivalent) with respect to the review and evaluation of teaching and papers, and; be supported by evidence from one or more of the review and evaluation methods stipulated in the Guidelines for the Review, Evaluation and Continuous Improvement of Teaching and Papers, one of which must be a Teaching Development Unit (TDU) teaching and paper appraisal; c. Maintain a portfolio of evidence of teaching and paper evaluations for Professional Goal Setting and promotions processes. APPRAISALS • TDU •8• 6. Chairpersons of department (or equivalent) are required, with respect to all teaching staff in their department (or equivalent), to: a. Ensure that a two year cycle of review and evaluation of teaching is implemented; b. Monitor evidence submitted in support of paper appraisals and teaching performance and development; c. Prepare an annual report to their Dean (or equivalent) on the teaching staff and papers reviewed within that calendar year; LY T N E Rannually HtoEtheD. 7. Deans (or equivalent) are required to report R T UFaculty/School/College’s Teaching Quality Committee on their N ATE C I implementation of this Policy. IS ND PD T N WA U E Compliance during Transitional PeriodG M E I N compliance with this Policy Ifull U 8. The University expects Vto achieve E C E B Quality Committee will monitor by theO end of 2010. R TheFTeaching D RprogressOtowards full compliance during the Sand support E I SS this will include identifying and addressing ND Eperiod, TH Utransitional relevantC issues including operational and resource implications. O PR d. Ensure that professional goal setting with staff addresses teaching and paper quality and evaluation. •9• APPRAISALS • TDU VI. Guidelines for the Review, Evaluation and Continuous Improvement of Teaching and Papers 1. Summative Tools 1.1 The Paper and Teaching Appraisal There are five main reasons for undertaking an appraisal: LY T To identify good features which should be N E maintained and developed further. R THE . R To identify unsatisfactory features U which N ATED I should be abandoned or modified. C D D IS Nneeds To determine professional development T P N U and goals in the light of the above. A Ethat Ewould W support G a M To gather information I N I U case for promotion or selection. C R EV BE O To gather required F for academic D information R O S audit E S I purposes. D H T UN ES OCan Appraisal 1.2 Initiating R P 1.2.1 WHO? An appraisal can be initiated by a paper convenor or staff member, who is the sole teacher for a particular paper, regarding their own paper and teaching. APPRAISALS • TDU • 10 • Note: When teaching a paper as a team it is preferable that the paper convenor or one of the teaching team request a paper appraisal and a teaching appraisal for each member of the teaching team. This avoids having multiple paper appraisal results for a given paper and reduces the number of questionnaires that students have to fill in. Please make sure that each member of the teaching team has given their consent for their teaching to be appraised. If individual members of the team do not want to be appraised at a particular sitting of the paper then they should be excluded from the appraisal. LY T N E R THE . R U IN TED 1.2.2 WHEN? C At the end of a paper: D DA IaSwholeNpaper T This allows for Preview and will Afor planning N U provide information next year’s E EW G M undertaking professional development, I EIN Upaper, V developing promotion/selection applications C E B audit procedures. O andRfor external F D S DER S O I Part-way a paper: Sallowsthrough TH UN E This for immediate fine-tuning of a paper C O and teaching and may have a direct pay-off for present students if modifications are made. PR However it should be noted that the open questions responses cannot be sent back until after the relevant Board of Examiners at the end of the semester. If immediate formative feedback is required it is suggested that an informal appraisal method is used. • 11 • APPRAISALS • TDU 1.2.3. HOW? The standard appraisal questionnaire is the University’s most basic form of appraisal questionnaire. It has eight standard summative questions and two formative open questions for the appraisal of the paper and eight standard summative questions and two formative open questions for the appraisal of the teaching. The standard questionnaire can be ordered by emailing or telephoning the Appraisals Administrator. This then has to be photocopied. Staff must be able to provide the full paper code (see below) and the relevant lecturer’s names. LY T N E R THE . R U IN TED C IS ND PDA Please note: T N WfourAparts: U Paper code: A paper code contains E G The department M Vcode E e.g. IPHIL250 I N U The E CyearRe.g.E07, B O The semester A, B, S or Y (see R and;OF S DDcalendar) E I S of the paper e.g. location S TH UN The E TGA (see calendar) HAM, C O code should look like PHIL250Rpaper So a full P 07A (HAM). Teacher’s name: Ensure that the teacher’s name is given in full e.g. Trudy Harris rather than T Harris. APPRAISALS • TDU • 12 • In some cases the standard appraisal questionnaire is not a suitable or subtle enough instrument to evaluate some aspects of the paper design or the teaching method. In these instances it is possible to produce a customised appraisal questionnaire. This can be done by adding questions from the item bank. 1. You can customise your evaluation by adding questions from the TDU item bank (see http:// www.waikato.ac.nz/tdu/customised.shtml). You can enter the items into the online order form on the website and either print it off and send it to me, or save and e-mail it to me. If this is a co/team-taught paper then you should consult your colleagues about the questions you would like to include. LY T N E R THE . R U IN TED C IS ND PDA T 2. If thereN is nothingA in the U item bank that will E W G provide adequate feedback about your paper M E I N I U V feel Efree to include some of your C then Eplease B own questions and submit as above. To this O R F think of the feedback that you please O S DDERend, I S like to have, both about the paper and TH UN ESwould your teaching. Again, if this is a co/teampaper then you need to consult with OC taught R your colleagues over the choice of questions. P If you choose questions from the item bank or wish for a customised evaluation then send the required information to the Appraisals Administrator who will produce a draft version of the appraisal. • 13 • APPRAISALS • TDU Please note: During the appraisal sessions at the end of each semester, it can take up to three days to generate a customised appraisal 1.3 What Appraisal Medium? Generally paper-based appraisals are still the norm, but recently there has been an increase in the number of web-based appraisals especially with the increase in online papers. The ordering processes for both mediums are the same. However the Appraisals Administrator will need to know the medium when a questionnaire is ordered. LY T N E R THE . If a paper-based appraisal is requested thenR the U IN TED questionnaire will be sent to the staffCmember concerned as a PDF file. This questionnaire Dwillto DA IS It N be ready for printing and photocopying. is up T P A N U the relevant convenor/staff member to copy it or E EW to G have it copied. M Please Irefer N http:// I U V E www.waikato.ac.nz/tdu/appraisal.shtml for C RE B guidelinesD onO photocopying questionnaires. F S DER S O I Ha web-based appraisal, a test link to the S TFor questionnaire willEbe sent to the staff member UN C who can test Oand submit data. This ensures that R the questionnaire works correctly and that the P information submitted to the Appraisal Administrator is in a proper format. Once the questionnaire has been approved, ITS moves the questionnaire to the live server. The live link is sent to the member of staff along with standard instructions for the students concerning anonymity of results. This complete process usually takes a couple of days, so please make sure that orders for online forms are not left to the last minute. APPRAISALS • TDU • 14 • 1.4 Conducting the Appraisal The TDU recommends that questionnaires be distributed at least one lecture prior to the final lecture, and not in the same hour as a test or assignment is set. It is also recommended that the appraisal questionnaires are distributed and completed at the start of the lecture. Prior to the students filling in the questionnaires, try to explain to them the purpose of the appraisal. Ask the students to fill in the questionnaire using blue or black ink, and to fill in the bubbles as opposed to ticking or circling them. Allow time for the students to complete the questionnaire; up to 10 minutes should be fine for a standard appraisal. The quality of the feedback tends to deteriorate if too little time is allowed for completing the questionnaire. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E Gthat the staff member It isM also recommended E I N I U the Eroom while students fill in the Cleaves EVteaching B O R questionnaires. The completed questionnaires F D R beO collected by a student representative or IS DEshould S staff volunteer. The completed questionnaires H S T UN E should be sealed in the appraisals envelope OC provided and sent to TDU either through the R internal post or placed in the appraisals box at: P TDU, A-Block Basement. A three minute DVD has been developed to inform students about the appraisals process: why it is done, what the information is used for and how to complete it. You may play this DVD in class to students. It is available at: http:// www.waikato.ac.nz/tdu/appraisal/index.shtml • 15 • APPRAISALS • TDU The teaching team should not view the completed questionnaires at this stage. The Privacy Act (1993) requires that the comments made by students are kept confidential. The questionnaires will be returned to those staff who requested them, after the relevant Board of Examiners meeting. 1.5 Processing the Appraisal For both paper-based and web-based appraisals the Appraisals Administrator will prepare an Appraisal Report of the summative information. This report is produced in a .PDF format and returned to the staff member via e-mail. Normally these results are ready within two to three weeks of the date that the completed questionnaires were sent to TDU. However at peak processing times these reports could take a little longer. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E G M willVIgenerally The Appraisal Report E INcontain U information relating the paper E and the C ontoRtheboth Epaper B O teacher. Information will be returned F D R to each teaching team. The E of tothe O IS member D information relatingSS the teacher will be H TreturnedUtoNthe individual CE teaching team member Along with these concerned. O questionnaires should R P be an Appraisal Cover To view the open questions responses for paper* based appraisals the staff member must have requested to view the original questionnaires on the information form. In line with the Privacy Act (1993) the original questionnaires will be sent to staff members after the relevant Board of Examiners. This is also true for those staff members conducting mid-semester appraisals. Staff may view the questionnaires for a month and they must be returned to TDU. During this APPRAISALS • TDU Sheet which should be signed by the teaching staff. This form gives information on how the teachers would like the data to be processed and whether they wish to view the questionnaires after processing. • 16 • time the staff member has responsibility for the confidentiality of the questionnaires. The paper questionnaires can be viewed by all members of the teaching team, while individual teachers only have access to their teaching questionnaires unless consent is given by other members of the teaching team. For web-based appraisals an Open Questions Report will also be produced but, as with the paper-based questionnaires, this report will only go back to the teaching team after the relevant BOE. LY T N E HEin Dall. Response rates need to R beRrecorded T appraisals. Where smallUgroupNappraisalsE are C must advise I TDU Tof the required, lecturers/tutors A S D I Dand tutorial students in each group. N Data onP lab T A N U sizes are not available through the student E(JadeSMS) W N database butGcan be provided by M E I I U Erates are needed at that level. Clecturers EifVresponse B O R F the original questionnaires for one R willOstore S DDETDU I Sfollowing the appraisal, after which time year S TH UN E they will be destroyed. C O student has the right to have PR Please note: Each their questionnaire destroyed after the report has been completed so that it is not available for viewing. • 17 • APPRAISALS • TDU 1.6 Interpreting the Appraisal Results The Appraisal Report shows the distribution of responses, on the relevant scale, for each question. Figure 1 shows a typical example. To the right of each graph a numerical table indicates the frequency of responses in the first column, with the total number of responses at the bottom. The middle column shows the distribution of responses in percentages, with the right-hand column showing the cumulative percentages. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E G M Report E Figure 1 A typical output from an Appraisal I N I U C REV BE O The “Overall, I was satisfied F with the quality of R O this Ipaper” provides staff with the S DDquestion E S H students perception of the quality of the paper. S TThe “Overall, UN CE this teacher was effective” O questions again gives staff a general indication of how effective PR the students believe their teaching Overall, I was satisfied with the quality of this paper Mean: 2.8 Always 1 3 5.6% 5.6% Usually 2 20 37.0% 42.6% Sometimes 3 20 37.0% 79.6% Seldom 4 8 14.8% 94.4% Never 5 3 5.6% 100.0% 54 100.0% 100.0% has been. 1.6.1 THE MEAN VALUE The means provided indicate the central tendency of the distribution. Taken in context with the scale used (Always, Usually, Sometimes, Seldom, Never) a mean of 1.6 for a paper questionnaire indicates a tendency for students to have responded that they were Always or Usually APPRAISALS • TDU • 18 • satisfied with the quality of the paper. A response of 3.4 on a teaching appraisal indicates that the students responded that the teaching was Sometimes or Seldom effective. 1.6.2 THE DISTRIBUTION PATTERN While the mean is a good indicator of the central tendency of the data, the distribution of responses in the histograms can also provide additional information. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E M VIE ING U C RE BE O F S DDER S O I TH UN ES OC R P A positive trend (Figure 2) is represented by most responses falling into categories 1–3. This is the trend that is hoped for. It indicates that, generally, all is going well and that the students think that the paper/teaching can be continued without change. However this does not mean that improvements cannot be made. Overall, I was satisfied with the quality of this paper Mean: Always 1 27 50.0% 50.0% Usually 2 Sometimes 3 16 11 29.6% 20.4% 79.6% 100.0% Seldom 4 Never 5 0 0 0.0% 0.0% 54 100.0% 1.7 100.0% Figure 2 A positive distribution of results A negative trend will have most responses in categories 3–5 (Figure 3). When this pattern emerges careful thought has to be given to why this response has been made for this question. Responses from the open questions could provide valuable insight into this result. • 19 • APPRAISALS • TDU Overall, I was satisfied with the quality of this paper Mean: Always 1 Usually 2 Sometimes 3 0 0 11 0.0% 0.0% 20.4% 0.0% 0.0% 20.4% Seldom 4 Never 5 16 27 29.6% 50.0% 50.0% 100.0% 54 100.0% 100.0% 4.3 Figure 3 A negative distribution of results Responses that are split between high and low ratings for the same question produce a polarised distribution (see Figure 4). This distribution could reflect student attitudes to changes in paper delivery, with those who are more traditional in outlook giving the negative bias and those who enjoy a more innovative approach providing the positive bias. Here the open questions responses would be most revealing. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E M VIE ING U C RE BE O F S DDER S O I TH UN ES OC R P Overall, I was satisfied with the quality of this paper Mean: 2.9 Always 1 Usually 2 24 5 44.4% 9.3% 44.4% 53.7% Sometimes 3 0 0.0% 53.7% Seldom 4 5 20 9.3% 37.0% 63.0% 100.0% 54 100.0% 100.0% Never 5 Figure 4 A polarised distribution The Open Question Report (generally for webbased appraisals) is a true recording of the student responses to the Open Questions in the appraisal. This information provides the rationale behind the student responses to the qualitative part of the appraisal and as such, provides valuable insight into the student’s experience. APPRAISALS • TDU • 20 • At the top of the Appraisal report information is provided on the whether the paper is sole, co- or team-taught and on student enrolment and response rate for a given paper. Information on the teaching team is taken from the Cover Appraisal Sheet (see page 3). Enrolment information is taken from the University’s most up-to-date enrolment query from BRIO. The information supplied is the overall enrolment for a given paper and does not give information on tutorial group sizes. The response rate indicates the number of questionnaires returned as a percentage of the overall enrolment figure. It is this response rate that is used in the Appraisal Summary mentioned on page 23. LY T N E R THE . R U IResults 1.7 Reporting of Appraisal N ATED C The reporting Iprocess S NatDeach Dlevel of the University isTindicated 1 below. All this P Ain Table N U information will be produced by E EW G the TDU. M IN for Demonstrators, U VI BAppraisals E PleaseEnote: C O Sessional R Assistants and Tutors will go to the F for distribution. D R O paper convenor S I DE S H N ES T Recipient U C Report O Teacher Paper and individual teacher report. PR Paper Convenor Paper report for that particular paper. COD Aggregated data for all papers relevant to department. Teaching reports through PGS. Dean/Faculty/School/ APC/TQC Aggregated paper results for each department in Faculty/ School. University/TEC Aggregated results for each Faculty/School. Table 1. The Reporting Structure within the University for Appraisal Information • 21 • APPRAISALS • TDU 1.8 Benchmarking Average results for department and faculties will be computed for use as benchmarks. These can be used by staff as a guide to how well their paper and teaching is developing. Generally, staff who have excellent teaching methodologies and paper design get a rating of 1 (which is consistent with Always satisfied with the quality of the paper; This teacher was Always effective). Staff can compare their results against Departmental mean values which will be distributed by their COD. Y L T 1.9 Appraisal Summaries N E Currently, Human Resources Policy requires an R THE . R Appraisal Summary for all promotion and U ED C increment reports. An Appraisal Summary is a IN T IS Nof D PDA two page document recording the statistics T A year.U each paper taught by a teacher the last NoverW E G Usually, this two-pageM limit is only enough space E I N I U V to provide statistical information for seven E C RE B papers. If D youO have taught more Fthan seven papers R O S over Ithe last year, choose the papers that you E then S D H S would like included in the T UN E summary. Appraisal summaries canC be ordered using the order form at O R http://www.waikato.ac.nz/tdu/appraisal/ P index.shtml APPRAISALS • TDU • 22 • 2. Evaluation for the Enhancement of Teaching To this point, the guidelines document has focused on the use of review and evaluation data for promotion and increment purposes (summative evaluation). An equally significant dimension of any review and evaluation of teaching and courses is to support the ongoing development of approaches to teaching (formative evaluation). The data set generated for promotion or increment purposes is generally quite different in scope, style and focus from that which emerges from a teaching development process. Thus, while it is unlikely that the same data set can be applied to these two quite distinct evaluation contexts, the one may inform the other. For example, summative evaluations of teaching and courses may signal the need for an individual staff member to be encouraged to place a greater emphasis on developing current teaching practice. Equally, a formative evaluation process may prompt a particular focus within a summative evaluation to ascertain the extent to which a particular teaching enhancement strategy had been effective. Even the most competent of teachers can further enhance their practice through reflective evaluations of the ways in which their approaches to teaching impact on student engagement with learning. The notion of reflective practice in teaching is summarised as LY T N E R THE . R U IN TED C IS ND PDA T N WA U E M VIE ING U C RE BE O F S DDER S O I TH UN ES OC R P ……..looking at what you do in the classroom, thinking about why you do it, and thinking about if it works - a process of self-observation and self-evaluation. By collecting information about what • 23 • APPRAISALS • TDU goes on in our classroom, and by analysing and evaluating this information, we identify and explore our own practices and underlying beliefs. This may then lead to changes and improvements in our teaching. Julie Tice, Teacher trainer, Writer, British Council Lisbon A number of formative tools can be usefully applied when considering the process of evaluation for development of teaching and a range of these is outlined in the following section of the document with, in addition, a representative bibliography at the end of this section. LY T N E R THE . R U IN TED C IS ND PDA T A U 3. Formative Tools EN W Gand E I N For evaluation U to M be comprehensive I V BE in the developmental, C it needs E to be embedded O R day to day of teaching F and learning. D context R O S ThereI are a number tools that can be E of formative S D H employed on a regular basis to inform teachers S Tand learners E UN about the quality of the learning Cindicate appropriate modifications O experience and PR and adjustments. In addition to using a range of tools, it is helpful for teachers to obtain feedback from different sources. Obviously, the learners’ perceptions are crucial, but research also indicates that student feedback on teaching and learning is limited. Incorporating other sources of feedback helps to provide a much richer picture of the teaching and learning. APPRAISALS • TDU • 24 • 3.1 Suggestions for Feedback from Students 3.1.1 Regular Anonymous Instant Feedback This can be done in a variety of ways. The aim is to provide quick feedback on a particular lecture, module or section of learning. These are simple, easy to obtain, read and analyse. Students may, for example, be asked to identify two key points: The aspect of the lecture/section/module that they found most interesting. The point of the class that they may find most difficult to understand or grasp. LY T N Another common strategy is R to E ask students to . HtheEmain R write a one sentence summary of T D U E learning in a particular C class. IANcomprehensive T A S D collection of in-class feedback exercises can be I Dexercises Nof these T P found in the compilation A N W(1993). This U feedback can bybe Angelo E and Cross G on-line. MandVhanded IE EinIorNposted written U CIt is R crucial E thatBthe lecturer responds to trends in O student F or revisits concepts that many D R feedback O S appear not to grasp. The ensuing I DEstudents S H conversations can extend and enhance the S T UN E classroom learning experience for both students C and teachers. O PR 3.1.2 The Critical Incident Questionnaire This is a particular form of instant feedback suggested by Brookfield (1995). This questionnaire, which lecturers are encouraged to distribute regularly to students, monitors the affective dimensions of the students’ learning experience as well as their cognitive understanding. Examples of the questions are: • 25 • APPRAISALS • TDU At what moment in the class this week did you feel most engaged with what was happening? At what moment in the class this week did you feel most distanced from what was happening? What about the class this week surprised you most? 3.1.3 Learning Reflection Exercises As part of the paper assessment students can be asked to write regular reflection exercises on their learning process. These may take the form of a learning journal or single reflection pieces. These exercises can also give lecturers an insight into the relation between their teaching and student learning. Additionally, reflection exercises help students to develop their metacognitive skills. LY T N E R THE . R U IN TED C IS ND PDA T N WA U E M VIE ING U 3.1.4 Focus Groups E way of C RE an excellent B Focus groupsOcan provide F student learning gaining anDin-depth RinsightOinto S E I and of teaching. S Small groups such as Dclasses Hexperience S N Tpostgraduate provide the optimum context U CEgroup. However, for using a focus group Ocan also be conducted focus discussions in a larger R P a small group of invited students when class with the teacher wants to examine some particular aspect of the paper or teaching more closely. Examples include investigating the students’ grasp of a particular theory or new teaching strategy. Focus groups must be conducted by a facilitator and a TDU staff member can facilitate a focus group discussion, document and analyse the findings. APPRAISALS • TDU • 26 • 3.2 Self-evaluation and Reflection 3.2.1 Regular Reflection In addition to student appraisal, an important part of professional development is ongoing reflection on and evaluation of one’s own practice. When evaluating a particular aspect of your teaching and assessment practices, useful criteria for evaluating your practices are: The intended learning outcomes LY T teaching and learning practicesN E E Your own experiencesR asR a learnerH T ED. (Brookfield, 1995).U C IN AT S I NDDiaryPD T 3.2.2 Keeping a Learning A U it is helpful to N andW To enhance sustain reflection, E G Ma diary E keep in which regularly jot down I N I U V BEof youryousense your impressions of the teaching C E O R and learning that has occurred. There are F D RnumerousOquestionnaires available to help you S I DE to askSprecise questions about specific aspects H S teaching, such as those provided by Brownof T UN Eyour OC and Race (1995), Forsyth, Joliffe & Stevens R (1995), Moon (1999) and Kahn & Walsh P Your teaching philosophy Research-based evidence on effective (2006). 3.2.3 Compiling and Maintaining a Teaching Portfolio Teaching portfolios are increasingly required for career development purposes, but they are also very important tools for professional development. In a teaching portfolio, teachers generally articulate their teaching philosophy and it evolution, document and explore elements of their current practice, and plan future • 27 • APPRAISALS • TDU teaching initiatives, research on teaching and professional development. The portfolio provides an ideal way for the University teaching to document and examine their pedagogy and practice and to frame future teaching-related choices. It is a flexible document so that shifts in thinking, new approaches and initiatives can be readily included. 3.3 Peer Interactions LY T N E R THE . R U IN TED C Peer lecture/tutorial observation using D DA IS negotiated criteria that have been carefully N T P and agreed (TDU staff can alsoAprovide N U Eand feedback). W NG teaching observation M E I I that U department V Bgatherings E Informal regular C E O to RdiscussFcurrent classroom inviteDpeople R colleagues. O experiences S DEwith I S H N ES departmental seminar T Teaching-focused U series. C RO Open lectures by experienced academics P followed by discussion sessions. Working with colleagues on our teaching can be very helpful. Sharing ideas about pedagogy and practice provides another important way of evaluating and developing your teaching. Contexts for peer interaction include: Using Moodle to exchange thoughts on ongoing teaching experiences. APPRAISALS • TDU • 28 • 3.4. Submitting Evidence Formative Evaluation of your The use of formative evaluation is an intrinsic part of the relationship that exists between the learner and the teacher. As such, examples of formative evaluation should only be submitted with PGS documentation to the COD. For help with Formative Evaluation please call the TDU. LY T N Teaching Developer (Quality/Evaluation) E Dr Trudy Harris R THE . R Rm AB.08 U IN TED C Tel: 07 838 4634 IS ND PDA E-mail: [email protected] T A U N [email protected] E W G M VIE forIN 4. Processes Continuous U C Improvement E BE O R F improvement of teaching and continuous O S DDERThe I S goes beyond the individual teacher. It S TH UN Elearning involves the whole department and ultimately C University. In addition to the Teaching O the R Quality Committee and the work of the TDU, P Departments and Schools/Faculty need to institute a review system which encourages continuous improvement and staff/student consultative committees as well as student representatives in individual courses to provide feedback. The aim should be a continuous process of feedback of information, mutual support and ultimate enhancement of quality. This goes beyond quantitative surveys and becomes part of the culture of the institution. • 29 • APPRAISALS • TDU 5. Representative Bibliography Angelo, T.A. & Cross. P. (1993). Classroom Assessment Techniques. San Francisco: Jossey Bass. Argyris, C.& Schon, D. (1975). Theory and P r a ct i c e; I nc r ea si ng pr o f e ss i o n al effectiveness. London: Jossey-Bass. LY T N E R THE . R U IN TED C Bell, M. (1996). Developing reflective Din DA IS practice N the education of universityT teachers. Paper P A U N published in the Conference Proceedings of E EofWthe Higher G M the Annual Conference I N I U Education C ResearchEV and Development E B O R Association of Australasia (HERSDA), Perth: F D R Western E SO IS Australia. D H S for quality learning at TBiggs,UJ. N E (1999). Teaching university. Buckingham: Open University. OC R P Bone, A. (1999). Ensuring successful assessment. Ballantyne, R., & Packer, J., (1995). Making connections. HERSDA, Gold Guide Number 2. Australian Capital Territory: Higher Education Research and Development Association of Australasia Inc. Coventry: Education. National Centre for Legal Boud, D. (1990). Assessment and the promotion of academic values. Studies in Higher Education 15 (1) 101-111. Boud, D. (1991). Implementing student self assessment. Canberra: HERDSA. APPRAISALS • TDU • 30 • Boud, D. (1995). Enhancing learning through self assessment. London: Kogan Page. Boud, D. (Ed). (1998). Developing student autonomy in learning. (2nd edition) London: Kogan Page. Boud, D., Keogh, R., & Walker, D. (1985). Reflection: Turning experience into learning. London: Kogan Page. LY T N E E . R TaHCritical Brookfield, S. (1995). Becoming R Reflective Teacher.: U San Francisco: Jossey ED N C I T Bass. IS ND PDA T A UAssess your own Brown, E S.N & Race, P. (1995) W teaching M Vquality. IE ELondon: NG Kogan Page. I U CBuzan, E BUse both sides of your brain. In O T. (1983). R F D O C. (1994). Processes for group S DERHogan, I S facilitators. Perth: School of Management and TH UN ESMarketing, Curtin University of Technology. C O W., & Kemmis, S. (1983). Becoming PR Carr,critical: Knowing through action research. Bright, B. (1995). What is reflective practice? Curriculum, 16 (2), 69-81. Geelong: Deakin University Press, Victoria, Australia. Claxton, G. (1999). Wise up: the challenge of lifelong learning. London: Bloomsbury. Cowan. J. (1999). On becoming an innovative university teacher. Buckingham: Open University. • 31 • APPRAISALS • TDU Elliot, J., (1983). Self-evaluation, professional development and accountability. In Galton, M., and Moon, B. (Eds), Changing Schools. London: Harper & Row. Forsyth, I., Jolliffe, A.& Stevens, D. (1995) Evaluating a course. London: Kogan Page. Fryer, R. (1997). Learning for the 21st century: first report of the National Advisory Group for Continuing Education and Lifelong Learning London: NAGCELL. LY T N Hall, S. (1997). Forms of reflective teaching E HE D. practice in higher education. In Pospisil, R. &R R T U IN TE Willcoxson, L. (Eds). Learning Through C Teaching, p124-131. Proceedings of the IS ND6th PDA Annual Teaching Learning T Forum, Murdoch AMurdochU N Perth: University, February E 1997. W University.http://lsn.curtin.edu.au/tlf/tlf1997/ M VIE ING U hall1.html. C E BE O R F of working D The explication Hall, S S. (1994). R O E I DwithinSaSteacher's self-evaluation N THknowledge E PhD thesis, Perth: Murdoch of her U teaching. C University, Western Australia. O R P Hall, S., Coates, R., Pearson, M., Ferroni, P. & Trinidad, S. (1997). Tilling the field: Action research in postgraduate supervision. In Pospisil, R. & Willcoxson, L. (Eds), Learning Through Teaching, p132-143. Proceedings of the 6th Annual Teaching Learning Forum, Murdoch University, February 1997. Perth: Murdoch University. APPRAISALS • TDU • 32 • Hinett, K. & Thomas, J, (1999), Staff guide to self and peer assessment. Oxford: Oxford Centre for Staff and Learning Development. Isaacs & Parker (1996). Short courses, beyond and beside: What do newly appointed university teachers want? Paper presented at (conference unknown), Queensland: Griffith University. Kahn, P. & Walsh, L. (2006) Developing Your Teaching. Oxan: Routledge LY T NA., Ng, S., Kember, D., Ha, T., Lam, B., Lee, E R TThe HEroleDof. Yan, L. & Yum, J. C. K. (1996). R U INaction Tresearch the critical friend in supporting E C projects. Published in the Conference A IS ND Conference D Proceedings of the Annual of the T P A N U Higher Research and Development E Education W G (HERSDA), Perth: Association of Australasia M E I N I U E C Western EVAustralia. B O R F R O S. & McTaggart, R. (1988). The action S DDEKemmis, I S planner. Third edition, Geelong: TH UN ESresearch Deakin University Press, Victoria, Australia. C O S. & McTaggart, R. (1996). The action PR Kemmis, research reader. Third revised edition, Geelong: Deakin University Press, Deakin University, Victoria, Australia. Martin, E. & Ramsden, P. (1994). Effectiveness and efficiency of courses in teaching methods for recently appointed academic staff. Canberra: Australian Capital Territory, Australia. • 33 • APPRAISALS • TDU McDrury, J. (1996). Developing tools for reflective practice. Paper published in the Conference Proceedings of the Annual Conference of the Higher Education Research and Development Association of Australasia (HERSDA), Perth: Western Australia. Mcgill, I. & Beatty, L. (1992). Action learning: A guide for professional management and educational development. London: Kogan Page. Y L T McGill, I. & Brockbank, A. (1998) Facilitating N reflective learning in higher education. E R THE . Buckingham: Open University. R U IN TED C Moon, J. (1999). Learning journals:Sa handbook I ND PDA for academics, students T and professional A U N Page. development. London:E Kogan W M VIE ING U Morrison, K. (1995). Dewey, Habermas C Curriculum, E B16E(2) 82-94.and reflective O practice. R D R OF S E(1996).SPreparing the university I Pettigrove, M. D H S N The E T teacher: PPIR cycle in context. Paper U Cat (conference unknown), presented O R Queensland: Griffith University. P Ramsden, P. (1992). Learning to teach in higher education. London: Routledge. Savin-Baden, M. (2000). Problem-based learning in higher education: untold stories. Buckingham: Open University. APPRAISALS • TDU • 34 • Schon, D. (1983). The reflective practitioner: How professionals think in action. London: Temple Smith. Schon, D. (1987). Educating the reflective practitioner. San Francisco, California: Jossey-Bass. Siksna, M. (1996, October). Reflective practice. Paper to accompany a workshop at the New South Wales/Australian Capital Territory regional staff developers' meeting, Armadale: University of New England, New South Wales. LY T N E R THE . R Slaney, K., Crawford, F.,U Parkin,NJ. & Taylor, EDP. C I T (1997). Independent thinking: Cross-cultural A SPospisil, DR. & Willcoxson, I D N possibilities. In L. T P A N U (Eds). Through Teaching, p296-302. ELearning W G6th Annual Teaching Proceedings of the M E I N I U Forum, E Murdoch University, C Learning EV 1997. B O February Perth: Murdoch University. R F D R IS DESmith,SR.O(1994). The experience of reflective H T UN ESuniversity teachers addressing quality in and learning. The CUTL action OC teaching R research project, Adelaide: University of P South Australia, Centre for University Teaching and Leaning. Smyth, J. (1991). Teachers as collaborative learners: Challenging dominant forms of supervision. In the series, Developing teachers and teaching. Day, C. (Ed). Bristol: Open University Press. • 35 • APPRAISALS • TDU Stenhouse, L. (1975). An introduction to curriculum research and development. London: Heinemann. Stewart, J., & Fantin, I. (1996). A case study of a research start-up program in higher education - changing the focus from teaching to research. Published in the Conference Proceedings of the Annual Conference of the Higher Education Research and Development Association of Australasia (HERSDA), Perth: Western Australia. Y L T Strauss, A. & Corbin, J. (1990). Basics of N qualitative research: Grounded theory E R THE . procedures and techniques. Newbury Park: R U IN TED Sage Publications. C D DA IS teachers. N Smyth, J. (Ed). (1987). Educating T A UP London: Falmer Press.EN W NG M E I I our own Uteaching: Vexploring Tice, J. Reflective E C E B O practice R Available: classroom http:// F D R ww ISw . t eDa cEh i n g eSn g lOi s h . o r g . u k / t h i n k / methodology/reflection.shtml H T UN ES Tripp, D. (1993). OC Critical incidents in teaching. R London: P Routledge. Teaching & Learning Centre, University of Victoria (1995). Critical incidents III: Legends of the fall term. A videotape & guidebook. Victoria: University of Victoria, British Columbia, Canada. Zuber-Skerritt, O. (1992). Action research in higher education: Examples and reflections. London: Kogan Page. APPRAISALS • TDU • 36 • 6. Contact Details Senior Lecturer Dorothy Spiller Rm A.B.09 Tel: 07 838 4466 ext 8697 E-mail: [email protected] Unit Administrator Preetha Pratapsingh Rm A.B.12 Tel: 07 838 4839 E-mail: [email protected] LY T N E R THE . Teaching Developer (Evaluation/Quality) R U IN TED Dr Trudy Harris C Rm A.B.08 IS ND PDA Tel: 07 838 4634 T N WA U E-mail: E [email protected] G M [email protected] E I N I U C REV BE O Administrator F D Appraisals R O Charlotte Ferry-Parker S E I S RmSA.B.07 TH UND E Tel: 07 838 4341 C O E-mail: [email protected] R [email protected] P Pro Vice-Chancellor (Teaching & Learning) Associate Professor Richard Coll KP.G.19 Tel: 07 838 4100 E-mail: [email protected] • 37 • APPRAISALS • TDU ……..looking at what you do in the classroom, thinking about why you do it, and thinking about if it works - a process of selfobservation and self-evaluation. By collecting information about what goes on in our classroom, and by analysing and evaluating this information, we identify and explore our own practices and underlying beliefs. This may then lead to changes and improvements in our teaching. Julie Tice, Teacher trainer, Writer, British Council Lisbon LY T Useful Links N E RDevelopment . HE Unit R The following documents can be found at the Teaching T D U E (TDU) website http://www.waikato.ac.nz/tdu C. IN AT S I ND PD T The Appraisal Order Form: N WA U http://www.waikato.ac.nz/tdu/appraisals/staff/apporder/ E M VIE ING apporder.shtml U C RE BE O Item Bank: F http://www.waikato.ac.nz/tdu/pdf/itembank.pdf S DDER S O I S Appraisal Questionnaires: N on Photocopying THGuidelines E U http://www.waikato.ac.nz/tdu/pdf/appraisals/guidelines.pdf OC R P Appraisal Summary Order Form: http://www.waikato.ac.nz/tdu/appraisal/summary.shtml Produced by: Teaching Development Unit Wāhanga Whakapakari Ako Private Bag 3105 Hamilton 3240 New Zealand Phone: +64 7 838 4839 Fax: +64 7 838 4573 [email protected] www.waikato.ac.nz/tdu
© Copyright 2026 Paperzz