Annual Assessment Report to the College 2010-­‐11 College: ____CSBS________________________ Department: __Political Science_____________ Program: __Undergraduate_________________ Note: Please submit your report to (1) the director of academic assessment, (2) your department chair or program coordinator and (3) the Associate Dean of your College by September 30, 2011. You may submit a separate report for each program which conducted assessment activities. Liaison: _Jennifer De Maio____________________ 1. Overview of Annual Assessment Project(s) 1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the program this year. Is assessment under the oversight of one person or a committee? Our department has been using a direct assessment model which we call Progressive Direct Assessment (PDA). The approach is designed to involve all faculty in the department, be an integrated component of the existing educational process, and provide information about student learning outcomes from students’ introduction to Political Science research methods to their final courses as majors in the department. For the current academic year, our department continued assessing our gateway and capstone courses. With these courses, our intention is to assess each of our departmental SLOs and track our students as they progress through the Political Science major. We received copies of final exams or papers from the following courses from Fall 2010 and Spring 2011: Political Science 372, Political Science Political Science 471C and Political Science 471D. Instructors were asked to provide the exam questions or essay prompts given to the students in order to provide context for the scoring. A sample of ten works per course were randomly chosen. This year, the assessment committee consisted of three full-­‐time faculty members including the assessment coordinator. We divided the committee into three teams of two members each who then read the exams or final works individually and scored them according to the SLO rubrics applicable to each course. The assessment committee members then reconciled the final scores on the exams/works and reported the results to the assessment coordinator. De Maio September 2011 1 The results were synthesized into charts (see attached) that show the percentages of works meeting the ‘below satisfaction’, ‘satisfactory’ and ‘exemplary’ designations. These were then distributed to the department. In 2010-­‐2011, we also participated in the Simplifying Assessment Across the University Pilot Program. Like our model, the Simplifying Assessment program asks departments to assess signature assignments from gateway and capstone courses. We have benefited from participating in the program and used what we have learned to further streamline our assessment activities. 1b. Implementation and Modifications: Did the actual assessment process deviate from what was intended? If so, please describe any modification to your assessment process and why it occurred. We had intended to assess the materials from our courses using rubrics we had uploaded to Moodle as per the Simplifying Assessment model. It turned out to be much easier to use hard copies of our rubrics as we had done in the past. We would still like to be able to use Moodle for Assessment so the committee will work on making the process more user-­‐friendly. We are also noticing a problem with 372 not necessarily being a gateway course. 2. Student Learning Outcome Assessment Project: Answer questions according to the individual SLOs assessed this year. If you assessed more than one SLO, please duplicate this chart for each one as needed. 2a. Which Student Learning Outcome was assessed this year? We assessed all six of our student learning outcomes: I. Professional Interaction and Effective Communication – Students should demonstrate persuasive and rhetorical communication skills for strong oral and written communication in small and large groups. II. Develop a Global Perspective – Students should demonstrate knowledge and theories relevant to global politics and policies. This includes knowledge of Western and non-­‐Western political systems, processes, values and models of politics and patterns of interaction among them. Students should demonstrate an understanding and respect for economic, socio-­‐cultural, political and environmental De Maio September 2011 2 interaction of global life. III. Active Citizenship and Civic Engagement –Students should demonstrate a knowledge and awareness of contemporary issues, political institutions, and problems in the community and their historical contexts. Students should demonstrate an understanding of the importance of community involvement and leadership. IV. Critical Thinking – Students should demonstrate increasingly sophisticated skills in reading primary sources critically. Students should be able to research and evaluate the models, methods and analyses of others in the field of Political Science, and critically integrate and evaluate others’ work. V. Political Decision Making– Students should demonstrate an in-­‐depth understanding and knowledge of the political institutions through which public policies are formulated, modified, and implemented. VI. Political Analytical Skills – Students should demonstrate a working knowledge of research designs, hypothesis formulation, measurement of variables, data collection, and analysis. 2b. What assessment instrument(s) were used to gather evidence about this SLO? We used our PDA model and collected ten samples of final exams or final papers from the following courses: Political Science 372, Political Science 471C, and Political Science 471D. Two members of the assessment committee were then asked to read the samples and score them following our assessment rubric. 2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of students, which courses, how decisions were made to include certain participants. We chose ten papers at random from the courses assessed. The average size of the classes assessed was 30-­‐40 students. The majority of the students assessed are political science majors. In the gateway (372) class, most students are in their 2nd or 3rd year while the students in the gateway courses (471) tend to be in their final year in the program. De Maio September 2011 3 2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-­‐sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used. We are looking at student learning as our majors progress through the program. We want to see what our majors are learning as they are introduced to the concepts of political science research and what they retain and how they develop with regards to critical thinking as they exit the program. 2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the collected evidence. Once the papers were read and scored by members of the assessment committee, the liaison used rubric tally sheets to record the scores and prepare graphs [see attached charts]. The results for the writing rubrics 1a and 1b are consistent across the classes. The majority of students scored in the satisfactory range. This suggests that there is still a need to work on the development of SLOs 1a & 1b. Students also appear to improve somewhat with regard to the other SLOs, especially 4a which measures critical thinking. Based on assessment data we have collected over time, the department suggested that faculty provide more detailed instructions to students as the more detailed the exam question/essay prompt, the better the student outcomes tended to be. This approach has been adopted by many faculty and the evidence suggests slight improvements in relation to previous years. 2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program, student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc. Please provide a clear and detailed description of how the assessment results were or will be used. The results of the 2010-­‐2011 assessment cycle will be shared with the department and be made available on our website. The curriculum committee will meet to discuss the report and begin to think about implementing changes based on the evidence. At this stage, we are planning to use the resulting evidence to recommend revisions to our undergraduate curriculum specifically with regard to the sequencing of our courses and to the consideration of course additions/deletions. We will also use the assessment data to help justify the need for new faculty positions. De Maio September 2011 4 3. How do this year’s assessment activities connect with your program’s strategic plan and/or 5-­‐yr assessment plan? We continue to collect data on student learning in order to track our majors as they progress through the program. We are now particularly interested in applying the evidence we have gathered to revising our undergraduate curriculum. We have already made important changes based on assessment data including ensuring that all instructors list the SLOs on their syllabi and give the students clear instructions for their final papers/exams. This year, the assessment liaison is serving on the curriculum committee and we hope that we will foster collaboration and a more direct application of evidence of student learning to program activities. Assessment is on the agenda at each of our department meetings and this year’s activities will become part of an ongoing discussion among faculty with regards to the development of writing, critical thinking, and political analytical skills among our majors. We are broadening our assessment program and are talking about creating an Introduction to Political Science course that would better serve as a gateway to our major. 4. Overall, if this year’s program assessment evidence indicates that new resources are needed in order to improve and support student learning, please discuss here. Our students are doing relatively well as they progress through the major. We would like to see more students scoring in the exemplary category, particularly with regards to writing and critical thinking. Discussions based on assessment evidence indicate that hiring new faculty would make an important and necessary contribution to student learning. We are not serving our students as well as we would like and in order to give our students the focused attention they need to develop their skills as political scientists, we need to increase the number of full-­‐time faculty. 5. Other information, assessment or reflective activities not captured above. For the 2011-­‐2012 assessment cycle, we have expanded the size of the assessment committee and are broadening activities beyond our major to track student learning in our Title V class. 6. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your program? Please provide citation or discuss. Cole, Alexandra and Jennifer De Maio. “What We Learned About our Assessment Program that has Nothing to do with Student Learning Outcomes.” Journal of Political Science Education, Vol. 5, No. 4, October -­‐ December 2009. De Maio September 2011 5 Academic year 2010-­‐2011 ASSESSMENT DATA [average] SLO 1a: Conventions and Coherence: Refers to the mechanics of writing such spelling, grammar and sentence structure. Includes stylistic considerations such as formatting and source documentation. 100%
80%
Exemplary
Satisfactory
Below Satisfaction
60%
40%
20%
0%
372fa10
372sp11
471cfa10
471csp11
471dsp11
SLO 1b: Rhetorical Aspects: Refers to the purpose of the assignment, organization of thoughts, development of an argument. 100%
80%
Exemplary
Satisfactory
Below Satisfaction
60%
40%
20%
0%
372fa10
372sp11
471cfa10
471csp11
471dsp11
De Maio September 2011 6 SLO2a: Demonstrate a Global Perspective: choose disciplinary theories, concepts, frameworks to organize and analyze 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
471cfa10
471csp11
SLO 3a: Active Citizenship and Civic Engagement: demonstrate the ability to conceptualize and specifically identify an important political and policy issue Exemp
Satis
Below
100%
50%
0%
471dSp11
De Maio September 2011 7 SLO 3b: Show various leadership styles and/or strategies SLO4a: Critical Thinking Present and support argument 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
372fa10
372sp11
471cfa10
471csp11
471dsp11
De Maio September 2011 8 SLO 4b: Identify pros and cons, analyze and evaluate alternative points of view SLO 5a: Political Decision Making: identify and describe political issues and institutions involved in solving these issues 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
471cfa10
471csp11
De Maio September 2011 9 SLO 6a: Political Analytical Skills: Identify research questions, propose hypotheses and analyzes, critique and integrate source material 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
371fa10
371sp11
471cfa10
471csp11
SLO6b: Identify sampling population, design instrument and collect data 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
372fa10
372sp11
De Maio September 2011 10 SLO 6c: Use appropriate method to analyze data 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
372fa10
372sp11
SLO 6d: Present data in an appropriate format 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
372fa10
372sp11
De Maio September 2011 11 SLO 6e: Present and accurately evaluate strengths and weaknesses of own research 100%
80%
Exemp
Satis
Below
60%
40%
20%
0%
372fa10
372sp11
De Maio September 2011 12