PROGRAM EVALUATOIN REPORT Counselor Education and Supervision Doctoral Program (Counselor Education Track) School of Counseling College of Health Professions The University of Akron Spring 2015 PROGRAM EVALUATION REPORT 2 Table of Contents Page Introduction…………………………………...……………………………………………. 3 Program Overview…….……………………………………………………………...……. 3 Program Evaluation….……………………………………………………………………... 3 AA.1……………………………………………………………………………………. 4 Program and Curriculum…………………………………………………………. 4 Program Applicants………………………………………………………………. 4 AA.2……………………………………………………………………………………. 5 Survey of Program Graduates……………………………………………………. 5 Survey of Current Students………………………………………………………. 6 AA.3……………………………………………………………………………………. 8 Survey of Supervisors/Employers………………………………………………... 8 AA.4……………………………………………………………………………………. 9 Student Candidacy Fitness Evaluation…………………………………………… 9 Annual Student Review…………………………………………………………... 10 Written Comprehensive Examination……………………………………………. 10 Program Signature Assessments…………………………………………………. 11 AA.5……………………………………………………………………………………. 11 Curriculum Modifications………………………………………………………….. 11 Program Objectives Revision………………………………………………………. 12 Revision of the Written Comprehensive Examination…………………………….. 13 Changes in Internship Requirements………………………………………………. 14 AA.6……………………………………………………………………………………. 14 Conclusion………………………………..………………………………………………… 14 PROGRAM EVALUATION REPORT 3 INTRODUCTION The purpose of the program evaluation report is to describe outcomes of a systematic program evaluation, with descriptions of any program modifications, to students currently in the program, program faculty, institutional administrators, personnel in cooperating agencies (e.g., employers, site supervisors), and the public. Within this report, a systematic program evaluation of the Counselor Education and Supervision Program—Counselor Education Track is described. The program faculty used data collected from 2012-2013, 2013-2014, and 2014-2015 academic years. Outcomes include objective and well-rounded faculty reflection about program strengths and areas for improvement, program-related transparency with stakeholders (e.g., students, graduates, and employers), and communication about the program’s status with institutional administrators. PROGRAM OVERVIEW The Counselor Education and Supervision Doctoral Program—Counselor Education Track has been accredited by the Council for Accreditation of Counseling and Related Educational Programs (CACREP) since 1985. The program is one of the six Counselor Education and Supervision doctoral programs in the State of Ohio and one of the two in Northeast Ohio. The program is housed in the School of Counseling, College of Health Professions. Currently, there are four full-time core faculty members in the Counselor Education and Supervision Program—Counselor Education Track: Drs. Delila Owens, Cynthia Reynolds, Robert Schwartz, and Varunee Faii Sangganjanavanich (program and clinical coordinator). In 2014-2015 academic year, there are 25 active students enrolled in the program. The program has produced 35 graduates since 2004. The Counselor Education and Supervision Doctoral Program—Counselor Education Track undergoes periodic reviews (every Spring semester) by the program faculty. These reviews are primarily focused on program improvements related to coursework, policies, and educational standards. As an accredited comprehensive doctoral degree program, programmatic changes are required to comply with national accreditation standards (e.g., CACREP), State licensure guidelines (e.g., Ohio Counselor, Social Worker, and Marriage and Family Therapist Board), College of Health Professions, and University policies. Programmatic changes are thoroughly reviewed by faculty members (and university administration and community professionals as needed) before implemented. PROGRAM EVALUATION The program faculty utilized the program evaluation criteria outlined in the 2009 CACREP Standards (Section I, Learning Environment: Structure and Evaluation, Evaluation, AA.1-6.) as a guide to systematic program evaluation. The below information is provided in response to the standards. PROGRAM EVALUATION REPORT 4 AA. Program faculty members engage in continuous systematic program evaluation indicating how the mission, objectives, and student learning outcomes are measured and met. The program faculty have engaged in continuous assessment of program strengths and areas for improvement in order to ensure that its mission, objectives, and student outcomes are met. The formal and informal program evaluations occur through regular faculty conversations, meetings, conversations with students, student/graduate/supervisor/employer surveys, and feedback from professionals in the field. The program evaluation data obtained lead to in-depth program reflection and updates including but not limited to the following areas: content published in the program handbook, curriculum, resources and marketing information (e.g., program brochures and websites), and training requirements and resources (e.g., practicum and internship). The data obtained during the most recent program evaluation process, and the program updates resulting from program faculty reflection, are presented below. AA.1. A review by program faculty of programs, curricular offerings, and characteristics of program applicants. Program and Curriculum In addition to continuous informal reviews, the Counselor Education and Supervision Program— Counselor Education Track faculty conduct a formal review of the program and its curriculum in Spring semester of every academic year. The findings of the reviews inform the program changes and improvement such as a revision of the curriculum, program objectives, and the written comprehensive examination, and changes in internship requirements. Detailed information regarding these changes can be found in AA.5. section in this report. To give an example, during the 2012 formal program review, there was a concern regarding dissertation writing and degree completion of students based on feedback from students and graduates. As a result of the review, program faculty took steps to address the concern. First, the program developed a new course specific to doctoral dissertation proposal (5600:726, required for all students) in order to provide students foundational knowledge and skills for dissertation writing. Second, students enrolled in this course are required to work closely (e.g., meeting regularly, submitting dissertation proposal draft) with their advisors in order to make progress toward dissertation proposal. Lastly, the program required students to successfully propose their dissertation proposal before entering their internship experience. Implemented in Fall 2013, these innovative changes were intended to help students make progress toward their dissertation writing and ultimately, their degree completion. Program Applicants The program offers one time admission per academic year (for Fall semester). The application deadline is January 15 of every year. Interviews generally occur the beginning of February and PROGRAM EVALUATION REPORT 5 applicants are notified the admission decision no later than March. The program attracts applicants from diverse backgrounds and clinical experiences from within the State of Ohio and across the nation. Program applicants are evaluated in various areas including their academic aptitude for doctorallevel study, previous professional experience, fitness for the profession and the program’s philosophy and objectives, oral and written communication skills, as well as potential for scholarship, professional leadership, and advocacy in the profession. The program admits 4-6 doctoral students per academic year. AA.2. Formal follow-up studies of program graduates to assess graduate perceptions and evaluations of major aspects of the program. The program faculty conduct formal annual surveys for current students and program graduates to provide their feedback to the program. To ensure confidentiality, the surveys are administered online. The surveys assess their perceptions of major aspects of the program in the areas of training and preparation. The summary of most recent survey results are listed below: Survey of Program Graduates The Graduate Survey was sent to graduates from The University of Akron Counselor Education and Supervision Program—Counselor Education Track between 2001-2013 in November 2013. Eight graduates (6 women and 7 Caucasians), ranging from 35-49 years old, responded to the survey. The survey comprised 16 items inquiring about the graduate’s demographics, program effectiveness, and areas for program improvement. The majority of the program doctoral graduates (n=5) were employed in the counseling profession. Within that group, three graduates were currently employed as full-time counselor educators. All, but one, graduates were licensed as Professional Counselor (LPC), Professional Clinical Counselor with Supervisor Endorsement (LPCC-S), or School Counselor (LSC), and half of the graduates were certified as a National Certified Counselor (NCC). In addition, all, but one, graduates were members of the American Counseling Association and its subdivision such as the Association for Counselor Education and Supervision (ACES). Using a likert rating, 1 (very dissatisfied), 2 (dissatisfied), 3 (somewhat satisfied), 4 (satisfied), and 5 (very satisfied), graduates rated the effectiveness of the doctoral program in Counselor Education and Supervision in providing them knowledge and skills in the area of research (M=3.50), teaching (M=3.13), Supervision (M=4.63), Counseling (M=4.63), and Leadership and Advocacy (M=3.63). Graduates rated satisfied to very satisfied with the program’s effectiveness in preparing them in the area of ethics (M=4.25), professional identity (M=4.38), and professionalism (M=4.25). However, they rated somewhat satisfied to satisfied with the program’s effectiveness in preparing them in social justice and advocacy (M=3.63). PROGRAM EVALUATION REPORT 6 Concerning the effectiveness of program operation, graduates (N=8) rated their satisfaction as follows: Areas of Program Operation Helping you make progress toward completion of your degree. Facilitating your professional and personal well-being The use of technology/instructional technology Counselor Education faculty receiving/accepting your feedback about the program Counselor Education faculty help and support when you have questions and/or concerns Counselor Education faculty advising availability and effectiveness Clear communication regarding guidelines, policies, and procedures (i.e., student handbook) Program didactic coursework Program clinical coursework The program's ability to offer courses Overall rating of the UA Counselor Education and Supervision Doctoral program (e.g., if a prospective student asked you) Mean 4.00 3.29 3.00 2.71 3.71 3.43 3.86 4.14 4.43 3.86 3.43 Six out of 8 graduates commented that support from the program faculty was the most positive aspect, whereas the conflicts among department faculty were the least positive aspect about the Counselor Education and Supervision Program—Counselor Education Track. When asked to identify the area for improvement, graduates noted that offering more guidance during the dissertation process and hiring faculty with strong professional counselor identity could improve the quality of the program. Surveys of Current Students Current Student Survey (2013) The Current Survey was sent to students who were actively enrolled in The University of Akron Counselor Education and Supervision Program—Counselor Education Track in November 2013. Twelve students, entering the program between 2009-2013, responded to the survey. The survey comprised 7 items inquiring about the students’ perceptions and satisfaction of the program effectiveness and areas for program improvement. Using a likert rating, 1 (very dissatisfied), 2 (dissatisfied), 3 (somewhat satisfied), 4 (satisfied), and 5 (very satisfied), students rated the effectiveness of the doctoral program in Counselor Education and Supervision in providing them knowledge and skills in the area of research (M=3.17), teaching (M=2.75), Supervision (M=3.83), Counseling (M=3.92), and Leadership and Advocacy (M=3.67). Students rated satisfied to very satisfied on the program’s effectiveness in preparing them in the area of ethics (M=4.33) and social justice and advocacy (M=4.17). However, they rated somewhat satisfied to satisfied on the program’s effectiveness in preparing them in the area of professional identity (M=3.75) and professionalism (M=3.67). PROGRAM EVALUATION REPORT 7 Concerning the effectiveness of program operation, students (N=12) rated their satisfaction as follows: Areas of Program Operation Helping you make progress toward completion of your degree. Facilitating your professional and personal well-being The use of technology/instructional technology Counselor Education faculty receiving/accepting your feedback about the program Counselor Education faculty help and support when you have questions and/or concerns Counselor Education faculty advising availability and effectiveness Clear communication regarding guidelines, policies, and procedures (i.e., student handbook) Program didactic coursework Program clinical coursework The program's ability to offer courses Overall rating of the UA Counselor Education and Supervision Doctoral program (e.g., if a prospective student asked you) Mean 4.00 3.29 3.00 2.74 3.71 3.43 3.86 4.14 4.43 2.86 3.43 Students commented that support from the program faculty/advisors, opportunities to co-present and co-publish with faculty, and smaller cohort size were the most positive aspects of the Counselor Education and Supervision Program—Counselor Education Track. A lack of pedagogical opportunities (e.g., co-teaching or teaching master’s counseling courses), the department’s inability to offer consistent coursework, and the delays in course schedule announcement were the least positive aspects about the program. When asked to identify the area for improvement, students noted that providing an advanced notice about course schedule and providing more pedagogical opportunities to students through their graduate assistantship could improve the quality of the program. Current Student Survey (2014) The Current Survey was sent to students who were actively enrolled in The University of Akron Counselor Education and Supervision Program—Counselor Education Track in December 2014. Nine students, entering the program between 2011-2013, responded to the survey. The survey comprised 7 items inquiring about the students’ perceptions and satisfaction of the program effectiveness and areas for program improvement. Using a likert rating, 1 (very dissatisfied), 2 (dissatisfied), 3 (somewhat satisfied), 4 (satisfied), and 5 (very satisfied), students rated the effectiveness of the doctoral program in Counselor Education and Supervision in providing them knowledge and skills in the area of research (M=3.67), teaching (M=3.22), Supervision (M=4.22), Counseling (M=4.11), and Leadership and Advocacy (M=3.67). Students rated satisfied to very satisfied on the program’s effectiveness in preparing them in the area of ethics (M=4.44), professional identity (M=4.22), and professionalism (M=4.22). However, they rated somewhat satisfied to satisfied on the program’s effectiveness in preparing them in the area of multiculturalism and social justice (M=3.44). PROGRAM EVALUATION REPORT 8 Concerning the effectiveness of program operation, students (N=9) rated their satisfaction as follows: Areas of Program Operation Helping you make progress toward completion of your degree. Facilitating your professional and personal well-being The use of technology/instructional technology Counselor Education faculty receiving/accepting your feedback about the program Counselor Education faculty help and support when you have questions and/or concerns Counselor Education faculty advising availability and effectiveness Clear communication regarding guidelines, policies, and procedures (i.e., student handbook) Program didactic coursework Program clinical coursework The program's ability to offer courses Overall rating of the UA Counselor Education and Supervision Doctoral program (e.g., if a prospective student asked you) Mean 3.89 3.22 3.67 3.44 4.00 3.89 3.78 3.78 3.78 2.89 3.78 Students commented that support from the program faculty/advisors, opportunities to co-present and co-publish with faculty, well-preparation in clinical counseling and supervision, and smaller cohort size were the most positive aspects of the Counselor Education and Supervision Program—Counselor Education Track. A lack of pedagogical opportunities (e.g., co-teaching or teaching master’s counseling courses), comprehensive examinations, the department’s inability to offer consistent coursework, and the delays in course schedule announcement were the least positive aspects about the program. When asked to identify the area for improvement, students noted that reconsidering the comprehensive examinations, providing an advanced notice about course schedule, and providing more pedagogical opportunities to students through their graduate assistantship could improve the quality of the program. AA.3. Formal studies of site supervisors and program graduate employers that assess their perceptions and evaluations of major aspects of the program. The program faculty conducted a formal annual survey for site supervisor of current students and employers of graduates to provide their feedback to the program. To ensure confidentiality, the survey was administered online. The survey assessed their perceptions of major aspects of the program in the areas of training and preparation. Survey of Supervisors/Employers In February 2015, the Supervisor/Employer Survey was sent to internship site supervisors of interns and employers of graduates from The University of Akron Counselor Education and Supervision Program—Counselor Education Track. Ten supervisors/employers responded to the survey. The survey comprised 6 questions inquiring about their perceptions and satisfaction of the program students and/or graduates as well as areas for program improvement. PROGRAM EVALUATION REPORT 9 Using a likert rating, 1 (very dissatisfied), 2 (dissatisfied), 3 (somewhat satisfied), 4 (satisfied), and 5 (very satisfied), supervisors/employers rated the knowledge and skill level of the interns/graduate in Counselor Education and Supervision in the area of research, teaching supervision, counseling, leadership, ethics, professional identity, social justice and advocacy, and professionalism. The results indicated that supervisors/employers (N=10) were satisfied to very satisfied with the knowledge and skill level of the program interns/graduates (see below). Areas of Program Operation Research Teaching Supervision Counseling Leadership Ethics Professional Identity Social Justice and Advocacy Professionalism Mean 4.10 4.80 5.00 5.00 4.80 5.00 4.89 4.78 5.00 When asked about their perceptions of overall quality the program interns/graduates using a likert rating, 1 (poor), 2 (developing), 3 (competent), 4 (excellent), and 5 (extraordinary), supervisors/employers rated excellent (n=6) and extraordinary (n=3). Supervisors/employers indicated that the program interns/graduates were well-prepared in teaching, research, supervision, clinical counseling, and professional identity; had strong professionalism; demonstrated real world experience; and were excellent candidates of choice for positions in their counselor education programs. When asked to identify the area for improvement, one supervisor/employers noted that the program should continue to increase diversity among students and faculty as well as provide an interprofessional experience to students. AA.4. Assessment of student learning and performance on professional identity, professional practice, and program area standards. The Counselor Education and Supervision Program—Counselor Education Track faculty assessed student learning and performance on professional identity and practice based on various assessment methods including candidacy fitness evaluation, annual student review, comprehensive examinations, and program signature assessments to ensure that the training produces desirables student learning outcomes required by the 2009 CACREP Standards. Student Candidacy Fitness Evaluation Each student in the Counselor Education and Supervision Program is reviewed annually using the Student Candidacy Fitness Evaluation form. This form is intended to provide feedback to program faculty and students regarding performance in key areas related to specific courses PROGRAM EVALUATION REPORT 10 (5600:710 – Theories of Counseling, 5600:702 – Advanced Practicum, and 5600:738 – Clinical Supervision II) deemed as program signature assessment points. The purpose is to track student development and performance as well as to offer remediation when needed throughout the program’s curriculum. The Student Candidacy Fitness Evaluation assesses student development and performance in the areas of professionalism, maturity, ethics, and academic performance. Annual Student Review The program faculty conduct the annual student review in Spring semester of every academic year. The annual student review is intended to be informative to students and faculty in regards to the student progress and performance in the program and the profession. Students first complete their professional and academic information through a secured web-based system and are later invited to an in person meeting with the program faculty to discuss their progress and how program faculty can help facilitate their professional development. The program faculty also review data from multiple sources include: a. Review of active students’ individual course grades b. Review of active students’ overall grade point averages c. Review of Student Candidacy Fitness Evaluations (completed by each faculty advisor in consultation with other program faculty). d. Review of clinical supervisor Practicum and Internship evaluation/feedback. These sources of evaluation/feedback were chosen to ensure summative assessment of clinical competence before graduation: 5600:702 – Advanced Practicum and 5600:785 – Doctoral Internship. e. Review of School of Counseling statement of expectations f. Review of individual written and oral comprehensive examination scores and pass rates In this review, students and faculty have an opportunity to review their academic progress (e.g., GPA, clinical and classroom performance), scholarship (e.g., presentation and publication records) and leadership (e.g., participation in professional organizations). Students also have an opportunity to ask questions and share concerns regarding their academic progress and professional accomplishments. Program faculty offer their feedback, suggestions, and/or assistance in order to help students meet their academic and professional goals. Written Comprehensive Examination The comprehensive examinations comprise the written and verbal portions. The students are required to successfully pass the written portion prior to the verbal portion of the exam. The purpose of the exams is to assess student’s knowledge in the areas of counseling, supervision, assessment, ethics, and professional identity. See more information regarding the exam in section AA.5. PROGRAM EVALUATION REPORT 11 Program Signature Assessments In addition to aforementioned assessment methods, to ensure ongoing formative evaluation of the 2009 CACREP standards, every Spring semester, the program faculty review and evaluate student learning outcomes through the use of program signature assessments including: 1. Student Candidacy Fitness Evaluation b. 5600:710 – Theories of Counseling (first year of program study) c. 5600:702 – Advanced Practicum (second year of program study) d. 5600:738 – Clinical Supervision II (third year of program study) 2. Comprehensive Examinations (Spring semester of the third year of program study) a. Written examination b. Oral examination 3. Dissertation Proposal (before entering Internship I) AA.5. Evidence of the use of findings to inform program modifications. Based on the findings from the program reviews (including program assessment by faculty and evaluative feedback from current students, graduates, and supervisors/employers), the Counselor Education and Supervision Program—Counselor Education Track has made the following changes: Curriculum Modifications Effective in Fall 2013, the program has utilized a new Program Course Distribution (PCD) which serves as a program plan for students. The new PCD reduces the number of semester credit hours (from 120 to 100 credit hours), removes irrelevant courses that do not pertain to counselor education and supervision, and adds new courses that correspond to the training standards of counselor educators and supervisors. The rationale for adding, removing, and/or replacing the following courses is to better fulfill the Doctoral Standards section in the 2009 CACREP Standards and to further equip students with cutting edge knowledge and skills required by the profession. The curriculum changes are listed below: Advanced Diversity in Counselor Education (5600: 728, three credits) has been added to the curriculum to replace Social-Philosophical Foundations of Education (5100: 705, three credits) to address diversity and multicultural issues that are specific to counselor education and supervision. Pedagogy in Counselor Education and Supervision: Theory and Practice (5600: 724, three credits) has been added to the curriculum to replace Emerging Technologies for Instruction (5100: 635, three credits) to provide doctoral students the pedagogical training that is specific to counselor preparation. PROGRAM EVALUATION REPORT 12 Doctoral Research Proposal in Counselor Education (5600: 726, three credits) has been added in order to assist doctoral students in preparing for their dissertation research proposal and grant writing. Qualitative Research I (5100: 744, three credits) has been made a required core course to replace Research Design II (5100: 716, three credits) in order to better provide training in qualitative research designs and methods. Program Objectives Revision Based on the review of the program objectives, the Counselor Education and Supervision Doctoral Program—Counselor Education Track faculty believed an update was needed in order to better align with the CACREP standards and the current practice of the profession. The revised program objectives are included in the current program handbook and are listed below: The purpose of the Counselor Education and Supervision Doctoral Program—Counselor Education Track is to facilitate the professional leadership roles of counselor education, supervision, counseling practice, and research competencies of students to advance the counseling profession. Aligned with the current standards of the Council for Accreditation of Counseling and Related Educational Programs (CACREP), doctoral students are expected to develop the identity of a professional counselor, clinical supervisor, counselor educator, and researcher. This identity is intended to promote competence and professionalism as students begin careers in public and private sectors in the counseling and counselor education profession. Upon completion of the program, students will be able to: 1. Demonstrate a clear and strong professional counselor identity, evidenced by professional leadership roles of counselor education, supervision, counseling practice, and research competencies expected of doctoral graduates; 2. Demonstrate knowledge, skills, and practices beyond the entry-level program requirements in all of the following: counseling, counselor education, clinical supervision, pedagogy, and leadership; and 3. Promote diversity through culturally relevant counseling, supervision, teaching, research, and professional service. During the program, students will be able to: 1. Gain foundational knowledge in professional counseling, counselor education, clinical supervision, pedagogy, and leadership; 2. Engage in advanced training in clinical counseling and supervision; 3. Develop scholarship, research, and writing skills through didactic coursework and doctoral dissertation; 4. Promote counselor professional identity, leadership, and advocacy skills; and 5. Develop professional competencies in clinical counseling, supervision, teaching, research, and/or leadership through Internship. PROGRAM EVALUATION REPORT 13 Revision of the Written Comprehensive Examination The feedback from students regarding the written comprehensive examination warranted the program attention in terms of the structure, format, and evaluative process. Therefore, since 2013, a semester prior to the exam, the students taking the written comprehensive examination are required to attend a meeting with the program coordinator to discuss the structure, format, and evaluative process of the exam. In this meeting, students also have an opportunity to ask questions or clarify information regarding the exam. Unlike a previous format of the written examination, students receive all questions at least 60 days in advance in order to have an opportunity to develop their answers to the questions. Then, students are required to complete their answers in person and in a time-limited environment in all exam areas. Concerning the evaluation of the written examination, in addition to using a blind review system, the faculty developed a grading rubric for each area of the exam in order to better inform students about the evaluation criteria and to objectively and thoroughly evaluate the students’ responses. Changes in Internship Requirements There are two changes made regarding the internship requirements of the program. First, students are now required to complete the written comprehensive examination prior to entering their internship experience in order to promote their likelihood in dissertation and degree completion. Second, as mentioned earlier in this report, the program requires students to successfully propose their dissertation proposal prior to entering their internship experience. Implemented in Fall 2013, these innovative changes are intended to help students make progress toward their dissertation writing and ultimately, their degree completion. AA.6. Distribution of an official report that documents outcomes of the systematic program evaluation, with descriptions of any program modifications, to students currently in the program, program faculty, institutional administrators, personnel in cooperating agencies (e.g., employers, site supervisors), and the public. In February 2015, the report was published on the program website and disseminated to parties involved including program faculty, current students, graduates, and supervisors/employers. CONCLUSION The Counselor Education and Supervision Doctoral Program—Counselor Education Track has made significant improvements over the past few years in order to achieve training excellence for its students and to promote the quality of its graduates. The overall findings from the systematic evaluation of the program were positive and led faculty to make informed decisions about the program. Based on the findings, the program has made major program changes including the training curriculum, program objectives, written comprehensive examination, and PROGRAM EVALUATION REPORT internship requirements. To monitor the quality of the program, informal and formal ongoing evaluations from stakeholders continue to be a part of program development. 14
© Copyright 2026 Paperzz