Student Learning Outcomes Handbook Cosumnes River College January 2008 edition Vision Statement To provide the best overall education in California’s community colleges, Cosumnes River College is committed to teaching excellence, student success, and educational leadership. Cosumnes River College is an innovative educational and community center that prepares and empowers students to realize their unique potential to transform their lives and contribute to a just and democratic society as global citizens. CRC strengthens the cultural, social, and economic well-being of the region through its development of an educated and engaged citizenry whose decisions are informed by intellectual inquiry and scholarship, a pluralistic world view, and respect for humanity and the environment. Mission Statement Cosumnes River College is an open access, student-centered, comprehensive community college that prepares students to realize their educational and career aspirations through exemplary transfer, general, and career education in an academically rigorous and inclusive environment. CRC also offers exceptional instruction in basic skills and English for non-native speakers, as well as a broad array of life-long learning, community service, and workforce development programs. Values Statement Affirmed by the Academic Senate and the College Planning Committee (6/23/08) Approved by the LRCCD Board (7/16/08) As a community of scholars and practitioners with a deep commitment to equity and social justice, CRC values teaching and learning excellence and high academic standards sustained by academic integrity, fairness and mutual respect, and an ethic of care. The College’s core values of cultural competence and diversity, continuous learning and assessment, exceptional student services, and innovation are central to our collegial and environmentally responsible academic setting. Contents I. Introduction Overview of SLOs at CRC (p. 4) • the outcomes cycle • levels of student learning outcomes • developing outcomes • assessing outcomes • using outcomes for dialogue and planning • reporting assessments • outcomes and assessment SLO Glossary (p. 13) II. Course and Activity SLOs Frequently Asked Questions (p. 17) Writing course/activity outcomes ( p.19) • defining course/activity outcomes • recording outcomes in SOCRATES Assessing course/activity outcomes (p.21) • assessment tools • documenting course outcomes assessment results Using the results of course and activity outcomes assessment (p. 26) III. Program SLOs Frequently Asked Questions (p. 29) Introduction: What is a “program”? (p. 31) • defining programs • program structure Writing program outcomes (p. 34) • defining program outcomes • recording outcomes in PrOF Assessing program outcomes (p. 36) • assessment tools • documenting program assessment results Using the results of program outcomes assessment (p. 39) IV. General College-wide and General Education SLOs Frequently Asked Questions (p. 41) Current general college-wide outcomes Current GE outcomes Assessing college-wide and GE outcomes Using the results of college-wide and GE outcomes 3 Note: This document is designed so that any section can be useful alone, without reading the entire document. Because of this a certain amount of repetition of information is intentionally present. Thank you for your patience with this redundancy. 4 Introduction Overview of SLOs at CRC The Outcomes Cycle: Student Learning Outcomes (SLO’s) state the major skills and abilities, as defined by CRC faculty, which a student will possess upon successfully finishing a course or a program of study at the college. These skills may be gained, and assessed, at any point during the course or program. SLOs are a way of identifying and explicitly stating what we consider most important for our students to learn. This process ties very directly to the core of teaching and reflects the expertise of the college faculty. When we teach, we are helping our students achieve the outcomes of the course. SLO “assessment” refers to the ways in we measure the success of students at achieving the measurable skills described by the SLOs. SLO assessment involves multiple ways of evaluating student learning: exams, yes, but also many other things - term papers, problem sets, presentations, performance evaluations, “practical” exams, lab reports, essays, journals, portfolios, creative projects, reflective essays, etc. SLO assessment can also involve the student feedback techniques that we’ve been calling “classroom assessment” – short responses from students that tell the professor how a particular lecture or lab is working. Outcomes assessment leads to ongoing dialogue and informs our planning. We measure what our students are learning, talk to one another about what we find out and then use that information to improve student success. This involves outcomes development that defines outcomes for our courses/programs/college, outcomes assessment, in which we assess student success on those outcomes, development of dialogue to help us analyze and share the results of that assessment, and responsive change that uses the results to plan and make improvements. The diagram below shows the cycle: RESOURCES DEFINE/REFINE Student Learning Outcomes DOCUMENT Process and Results MAKE Changes (Instruction, Curriculum Program, Etc.) PLANNING DESIGN Assessment Tools THE OUTCOMES ASSESSMENT CYCLE Share findings with others and IMPLEMENT Assessment Tools to Gather Evidence ANALYZE and Evaluate the Collected Data IDENTIFY Strengths and Gaps 5 Levels of Student Learning Outcomes College faculty define and assess student learning outcomes at the course/activity, program, and college levels: Course outcomes describe what we, the faculty, have defined as most important for students to learn through our classes. Our course outcomes state the measurable skills and abilities that form the basis of student success in our classes. They are rather like course objectives, but are broader in scope and fewer in number. Course outcomes are defined and assessed by the teachers of the courses; they are formally stated on the course outlines and typically included on class syllabi. Assessment of student success on course outcomes is used to improve teaching, promote dialogue among colleagues, and support planning and resource requests related to course delivery. Activity outcomes are similar in many ways to course outcomes, but reflect the skills and abilities gained by students participating in college activities other than courses (student government, counseling activities, use of the library, athletic teams, etc.). They are defined by the faculty and staff who conduct the activities and are stated on the Program Review documents (PrOF) for the program. Assessment of activity outcomes is used to improve the way activities support student learning, promote dialogue among colleagues and support planning and resource requests that are related to the activities. Program outcomes reflect the cumulative result of learning throughout an entire program of study. The “program” could be the set of courses leading to a degree or certificate. However, there may be programs that do not lead directly to degrees or certificates. For example, “Counseling” can be defined as a program with respect to being able to identify student outcomes, but is not associated with any particular degree or certificate. Outcomes are defined by the faculty of a program. The assessment of program outcomes may occur across the courses/activities of the program, in “capstone” courses or activities, or through studies of program completers. Program outcomes are stated, and the results of outcome assessment are reviewed and discussed in the program review documents (PrOF). Assessment of program outcomes is used to promote dialogue, improve the program courses/activities and support planning and resource requests from the program. College outcomes reflect the overall college experience. They describe the major skills and abilities that a successful student will take with them “out there” as a result of their time here. This includes both overall college outcomes and more specific outcomes defined for the areas of general education at the college. The assessment of college outcomes will involve class-embedded assessments, institutional measures, and surveys. In large part, it will be done with the assistance of the college Research Office. The results will be used to promote dialogue, improve processes across the college, and support planning. 6 Summary of the different levels of outcomes assessment Course or activity outcomes Program outcomes Tools Course-embedded assessments: • analysis of student work on assignments, exams, quizzes • direct observation of skills through projects, creative products, performances, discussions, • student self-reflections such as journals, surveys, etc. Course-embedded assessments Capstone projects, portfolios, etc. Dialogue Professors teaching sections of the course or leading the activity via formal and informal meetings Department/Program faculty and staff via meetings and in PrOF Curriculum committee via course outlines College community via PrOF, etc. Department/Program faculty and staff via department meetings and in PrOF Response Improve methods that support student learning in classes and/or activities outside of the classroom Inform curriculum revisions and changes in related processes Inform classroom or activity related resource requests Inform PrOF and unit planning Improve processes and practices, both in and out of the classroom Inform department planning and curriculum review Division via unit plan College outcomes Inform PrOF and unit planning Surveys of program completers and alumni follow up studies Planning committee via PrOF Passing rates on National Certification Exams College community via PrOF, etc. Inform college strategic planning Research office studies Course-embedded assessments Professor Improve processes and practices, both in and out of the classroom Inform program resource requests Department Overall college outcomes Surveys of program completers Planning committee via PrOF Inform unit planning Institutional measures GE outcomes Research office studies Inform department planning and curriculum review College community via PrOF dissemination, etc. Inform college strategic planning Inform GE course review process of curriculum committee Inform review of College-wide outcomes by Academic Senate 7 Developing outcomes Faculty are the primary developers of student learning outcomes. The determination of learning outcomes for courses and instructional programs is the purview of the faculty teaching in those courses or programs. CRC staff other than faculty may provide useful input into the development of activity SLOs. For example, the development of activity (not course) SLOs, particularly in administrative service areas, may rely mainly on staff other than faculty. Student learning outcomes focus on the big picture rather than the details of tasks. SLOs describe the major goals of the student’s experiences at the college. For courses and programs, outcomes identify the things that the instructors feel are the most important skills that a successful student will take away from the educational experience for use in his/her future life. SLO’s are written to describe measurable skills and abilities. SLOs focus on what the students can do, that is, how they can apply the knowledge gained. The knowledge gained during the course, activity, or program forms the underlying basis of these skills and abilities. Checklist for SLO development: Was the outcome developed via dialogue among those faculty associated with the course, activity, or program? [Note: some activity outcomes may be developed mainly by staff other than faculty, e.g. in administrative services.] Is the focus on the big picture – a fundamental result of the course, activity, or program? Does the SLO describe what the student can DO with the knowledge they have acquired? Can the outcome feasibly be assessed? Do the SLOs represent skills and abilities that the students need to carry with them to the next stages of their lives and education? Is the language of the SLO clear enough so that it can be used by all of the faculty or staff associated with its implementation? Are the SLOs broad enough to support the academic freedom of individual professors to develop the details of the work to be completed in their classes, activities, and programs? Student learning outcomes for courses are listed on the course outline. Usually a course will have only a few SLOs –often between two and five. Some courses, e.g. in occupational areas, may have more SLOs due to the requirements of an advisory committee or accrediting body. Each SLO that is on a course outline will normally be inclusive of two or more course objectives (which are already in SOCRATES). Other sections of the course outline, especially the Methods of Instruction, Methods of Evaluation, and Typical Homework sections, should clearly align with the course outcomes. The Curriculum Committee will provide additional information on the expected formatting for SLOs in the course outline. Checklist for entering course SLOs into the course outline Do the Methods of Instruction, Methods of Evaluation, and Typical Homework sections of the course outline clearly link to the SLOs? Is enough information provided in these sections so that it can guide the teaching of a course or the writing of a syllabus? Is there enough flexibility for individual professors teaching the course? 8 Student learning outcomes for programs are stated in the program review documents (PrOF). A “program” is a set of courses or other activities that can be aligned with a cohesive set outcomes; it need not result in a degree or certificate. Each program typically have several (perhaps 4-6) student learning outcomes. The students achieve the program SLOs through the aggregate experiences of the program activities. The program SLOs should answer the question, “What will the students be able to do out there based on what they have learned here?” Objectives can be listed under each SLO to provide details. Program outcomes should clearly map to the courses and activities of the program and this alignment should be evident from the PrOF documents. The structure of the program will determine if program outcomes are reflected in each course/activity or if some program outcomes map to only one course/activity. However, the curriculum of the program should be cohesive; that is, all program outcomes should be clearly supported by courses and/or activities. The PrOF reporting documents are currently under review – look for updated information about how to format program SLOs from the CRC Planning Committee. Checklist for stating program SLOs in the PrOF Do the courses and activities in the program clearly map to the program outcomes? Are all program outcomes clearly supported by courses and/or activities? Have the formatting guidelines for PrOF been followed? Can the outcome feasibly be assessed for students currently in the program and/or for students who have recently completed the program? Assessing outcomes Assessment of outcomes can include both course or activity embedded assessments and broader assessments based on institutional tools. Course or activity embedded assessments occur as part of a class or an activity and focus on what the students learn during that class or activity. Broad institutional assessments focus on outcomes that aggregate over more than one course or activity (e.g. college-wide outcomes) and are implemented using institutional tools such as surveys conducted by the college research office. The determination of appropriate methods of assessment of student learning outcomes for courses is the purview of the faculty teaching those courses. Exams are recognized as an important part of the appropriate assessment of student learning, and many other methods of assessment are also appropriate assessment of student learning e.g. term papers, problem sets, presentations, performance evaluations, “practical” exams, lab reports, analytical essays, reflective essays, journals, portfolios, creative projects, etc. Informal classroom assessments, guided discussion, and other non-graded assessments may also be used to provide outcomes assessment information. Course-embedded assessment of outcomes uses many of the same tools as assessing student work for grades, but these processes are not the same. Assessment for grades evaluates the work of individual students across the entire set of course outcomes. Assessment of outcomes evaluates the work of the entire set of students on an individual outcome. The same tool can be used simultaneously for both purposes, for example designing an exam so that item analysis can yield outcomes assessment. 9 Assessments should be unbiased and show validity and be feasible to implement. An assessment is unbiased if it measures the same thing for all groups. An assessment shows validity if it measures the outcome it is intended to measure and not some other, possibly covarying, factor. Of course, assessments must be feasible; that is, they should not add an unacceptable amount to the workload or cost to the faculty or staff. Assessment of college-wide outcomes may be conducted through the agency of the college Research Office. Some outcomes, such as the broad college-wide outcomes, accrete gradually over the course of a student’s experiences at CRC and are very difficult to assess in any one course or activity. For these outcomes, the college Research Office can be useful in developing and analyzing assessments such as surveys, studies of alumni, etc. Faculty use outcomes assessment to judge how well the students are learning; the outcomes assessments are not used to judge the professors. Student Learning Outcomes are appropriately used as both self-referential evaluative tools and tools for evaluating student performance. This means that faculty evaluate the students’ learning and use this information as appropriate. Checklist for developing outcomes assessments Do the assessments clearly link to the SLOs? Do the assessments show reliability and validity Are the assessments feasible? Can they be implemented within a reasonable workload and minimal cost? Are multiple measures for assessment listed (when appropriate)? Is it clear that the results of the assessments will be used to improve the course, activity, or program but not to judge the professor? Using outcomes for dialogue and planning Dialogue between colleagues about student learning is an essential component of the implementation of SLOs at CRC. Dialogue about student learning and discourse about “best practices” both inside and outside of the classroom is a professional activity for CRC professors and other college staff. This dialogue may be informal discussion between colleagues (e.g. those great “hallway conversations” we all have) or it may involve more formal processes such as attendance at CASSL workshops, departmental meetings, events sponsored by the Professional Development Committee, etc. Analysis of outcomes assessment data is used to make changes that support student learning. When we change our teaching methods because of what we understand about our student’s learning, we “close the assessment loop”. This could include many types of change such as requests for resources, curriculum actions, budgetary changes, changing information communicated to other departments, changing advice given to students, disseminating information about “best practices” to departmental faculty, modifying the mentoring of new staff and adjunct faculty, implementing new teaching strategies, etc. 10 Implementation of SLOs at CRC is a central part of college planning processes. Outcomes assessment allows us to plan changes that will help our students succeed. The results of our outcomes assessments inform our planning and resource requests. Our PrOF planning sections should link to our outcomes analyses. Our Unit Plans are, in turn, based on the planning described in the PrOF. These plans guide our resource requests. The college strategic planning process, in turn, incorporates what we learn through our outcomes assessment processes. Diagram of Outcomes Assessment Flow Outcomes Assessment Processes A. Research Office surveys of college-wide and GE outcomes (student self-assessment) B. Faculty and staff reports of course or activity embedded assessments Local Planning Unit Plans Program Review documents (PrOF) dialogue Dialogue Departmental & Area dialogue Integrated analysis & College Institutional Planning College Planning Shared Governance planning Responsive Change Changes in practices, resource use, professional development, etc. 11 Reporting assessments Overview: The Outcomes Assessment Taskforce is currently developing processes for reporting assessments. From these discussions we have identified a proposed structure for outcomes assessment at the college. This structure is based on two types of assessment: Surveys conducted by the college research office and assessments chosen by individual faculty and staff. It includes: o Research Office surveys for college-wide outcomes: College wide outcomes will be assessed periodically through the Research Office using a student self assessment survey and a sampling methodology. Results will be shared via a college-wide dialogue with the campus community and will be discussed with the Academic Senate. o Course or activity embedded assessments of issues identified by the college or the program as critical: Each year programs will each identify a critical issue for assessment. o Faculty from programs already involved in detailed assessment and reporting procedures due to outside accreditation will not be required to duplicate these efforts (although a mechanism to integrate these efforts with the college's processes will need to be developed). Annual Reporting Cycle Processes for reporting assessments have been developed by the Outcomes Assessment Taskforce. The key part of the process is that program faculty and staff identifies the most important learning issues affecting the success of their students and focuses their assessment reporting efforts on outcomes related to those issues. Brief reporting forms will be collected each semester. The forms will be collected by the OATF. Samples of the OATF forms can be found in the appendix to this handbook. The information will be collated for use in program review, planning processes, and collegial dialogue. In Fall 2008, we will implement these processes for the first time on a college-wide basis. The process consists of two stages in each academic year: Fall Semester: At the beginning of the fall semester programs will be asked to identify the most important learning issues facing their students and the outcomes that align with those issues. More than one issue may be identified. Programs will focus their outcomes assessment reporting for the year on those outcomes. These may be college-wide, GE, program, course, or activity outcomes. Each program will complete a brief form that describes the following: • the important issues identified by the program faculty and staff • the outcomes associated with those issues • a summary of plans to assess, and report the assessment of, those outcomes Spring Semester: In the spring programs will report on their assessments of the outcomes identified in the fall. They will complete a brief form that reports the following • the important issues and related outcomes identified in the fall • the methods used to assess those outcomes • the results of the assessments • what was learned from that assessment; i.e. what was identified as effective and what can be changed to improve student success based on the fall assessment 12 Some important notes: o The faculty and staff of a program use their best professional judgment to identify learning issues important to the success of students in the program, to determine what outcomes reflect those critical issues, and to design assessments of those outcomes. o A subset of course SLOs can be assessed and reported each year; not all of the course SLOs need to be assessed and reported annually o One or a few program SLO(s) may be the focus of assessment one year; not all of the program SLOs need to be assessed annually. o SLO assessment does not require any specific research design; assessment may be based on narrative accounts, descriptive information and/or quantitative data and will often include multiple measures. o SLO assessment in instructional programs should be conducted for a sample of classes or students in a course, program or service area; however, SLO assessment does not mandate any minimum number of sections analyzed, does not require a particular method of assessment, and does not specify a minimum sample size. o An instructor is encouraged to save a record of the assessment privately to be able to describe the assessment to faculty in a department or program area. SLO assessment does not require the identification of individual instructors’ data in any reports or analyses. Student Learning Outcomes (SLOs) complete the statement “students completing program activities will be able to…” . Service Area Outcomes (SAOs) complete the sentence “In order to support learning, CRC employees will…” Outcomes and accreditation The new accreditation standards emphasize: • Documentation of a dialogue about outcomes development • Evidence of outcomes assessment and a dialogue about the results of that assessment. • Evidence of responsive change that closes the loop between what we learn from outcomes assessment and what we do in our classes and activities. The accreditation team will look for evidence (qualitative or quantitative) that outcome assessment occurs on an ongoing basis across the college. However, while outcome assessment is ongoing, we do not need to collect and store reports & documentation of the assessment of every outcome, at every level, every semester. Evidence of our dialogue with respect to outcomes development and assessment might come from a variety of sources. The accreditation team will look for evidence of the processes used, samples of student work showing assessment of outcomes, documentation of the dialogue involved, and examples of the tangible changes we’ve made in response to outcomes assessment. For example the accrediting team may look at: • information from program review (PrOF) documents • samples of work from classes • minutes from department meetings, committee meeting, etc. • email strings from the CASSL, or other discussions • examples of changes to course materials (e.g., syllabus, text, assignments, etc.) • examples of professional development opportunities sought out and/or attended • examples of resource requests based directly on outcomes assessment • results of projects such as the Education Initiative projects, classroom research, etc. 13 Much of what we learn from assessing outcomes is directly useful for us and our students. While the documentation and reporting are not all directly useful to us, there is a great deal of overlap between what accreditation requires and what is directly helpful to student success. The diagram below shows one view of the overlap. Required for a accreditation Records of data analyses Evidence of assessment Documentation of dialogue Evidence of responsive change Direct effect on student success Dialogues about learning Clear learning goals Shared best practices Planning focused on student success Effective teaching practices Collegial mentoring Curriculum development Effective resource requests 14 SLO Glossary General SLO Terms College SLO: A general statement of the skills and abilities a student will possess upon successfully graduating from a college. Examples: • General Education: The student will be able to utilize the basic methods of the discipline studied and apply those tools to personal and community issues. • Cultural competence: The student will be able to undertake thoughtful consideration of divergent points of view and utilize multiple perspectives in considering information. Course or activity SLO: A general statement of the skills and abilities a student will possess upon successfully finishing a course. Examples: • Non-majors biology course: The student will be able to draw reasonable conclusions from scientific data and apply the information to personal and community issues. • Computer applications course: The student will be able to choose the appropriate software application to use in a variety of situations and will use that application with precision. • Counseling appointments: The student will be able to complete a Student Education Plan and modify that plan as needed. Mapping (alignment) grid: A mapping or alignment grid is a table showing how parts of a course of study fit together with one another. Such a grid may include course objectives, student learning outcomes, methods of instruction, methods of evaluation, course themes, etc. An example of a table that could be used to show which courses/activities have major or minor emphases related program outcomes is shown below: SLO supported by course or activity (major or minor component): Course 1 Course 2 Course 3 Activity 1 Activity 2 SLO 1 SLO 2 Major Minor SLO 3 Major Minor Major SLO 4 Major Minor Major Major Minor Objective: More narrowly focused than an SLO, course objectives are directed toward specific tasks. Course objectives can be found on our course outlines. As do outcomes, objectives also finish the sentence “Upon successful completion of this course the student will be able to…” • Examples: Biology course: solve problems in Mendelian genetics using Punnett Squares. • Computer applications course: enter data into Excel and produce clear graphs. Program SLO: A general statement of the skills and abilities a student will possess upon successfully finishing a program. Examples: • AS in Psychology: The student will be able to analyze patterns of human behavior, developing thorough and complex explanations consistent with the major psychological perspectives. • Tutoring Program: Students who utilize tutors will increase success in basic skills courses as measured by improved scores on assignments and/or exams. 15 Student Learning Outcome (SLO) - An SLO is a general statement of the measurable skills and abilities a student will possess upon successfully finishing a course of study. It includes or refers to a plan to incorporate feedback and “close the loop” by changing the course of study. Broader in scope than “objectives”, SLOs focus on the important general abilities a successful student will take away from the class, program, or college for use in his/her future life. Taxonomies: In the context of SLOs, a taxonomy is a classification of skills and abilities based on level of achievement. The levels move from very simple processes to very complex ones. • Cognitive taxonomy: A classification of cognitive skills and abilities ranging from relatively simple tasks such as recognition to very complex processes such as critical analysis. • Psychomotor taxonomy: A classification of physical (motor) skills and abilities ranging from relatively simple imitation to creative mastery of a physical skill. • Affective taxonomy: A classification of skills and abilities that relate to values and emotions; levels range from relatively simple recognition of value to complex processes such as critical evaluation of ethical situations. Outcomes Assessment Terms Assessment of outcomes - Outcomes assessment is a process by which we evaluate our students’ mastery of the skills and abilities learned in their college activities, analyze the results of that process, and use those results to improve the learning environment for future students. Assessment loop - The assessment loop is closed when we use the results of our assessments to improve what we do in the future. Bias – Bias occurs if an assessment measures different things for different groups of subjects or has different levels of validity for different groups. For example, if an assessment depends on background knowledge (unrelated to the course material) that is possessed by some groups and not others, it may produce biased results. Direct assessment: This includes (1) any assessment tool that involves directly observing students as they utilize the skills taught in the class and (2) evaluation of student work that was produced using the skills taught in the class. For example, students in an Art class may be assessed based on their drawings, those in a Communication class on their public speaking performances, those in a Biology class on inquiry project work. Formative assessment - Formative assessment is conducted early in the learning process when mastery of the skill or concept is not yet expected. The purpose of formative assessment is to allow the student to improve performance on similar learning tasks in the near future. Multiple measures – Many outcomes can be assessed using a wide variety of tools (exam questions, papers, assignments, direct observation of student skills, portfolios, creative work, etc.). Typically, the use of multiple measures to assess an outcome increases the validity of the assessment. 16 Reliability – This reflects the degree to which an assessment would give the same results if given to a matched group of subjects at another time. Rubric -A detailed description of the criteria that will be used to grade students’ work or activities and the grading scheme based on these criteria. Analytical rubric: A rubric listing the traits on which the work will be evaluated each of which is associated with a specific number of points. The score on each trait will be added to produce the overall grade for the work or activity Holistic rubric: A rubric that provides holistic descriptions of the type of work producing each grade level (A-F) Self assessment – Measures the students’ own assessment of their knowledge or ability. This may include surveys, focus groups, discussions, brief classroom assessments, etc. Summative assessment - Assessment that is used to evaluate “completed” learning on a single objective, a group of tasks, or the overall course. Summative assessment occurs after the learning process for the given task is essentially complete and mastery of the skill or concept is expected. This type of assessment forms the basis for the grade in the course. Criteria-referenced summative assessment - In criteria-referenced evaluation the grade is based on previously criteria specified and is not affected by the score distribution within the group (i.e. the grade is not "curved" to the class norm). Norm-referenced summative assessment - In norm-referenced evaluation the grade is based on the score distribution within the group; also called "curved” grading. Validity – This reflects the degree to which an assessment measures what it is intended to measure and not some other, possibly co-varying, factor. For example, a timed exam may not be ‘valid’ if it measures the student’s ability to perform a task quickly when it was not intended to measure this (but was intended, instead, to measuring the student’s overall understanding of a subject). Accreditation Documentation & Reporting Terms ACCJC (WASC) – The organization that accredits community colleges in the western United States--including CRC. This organization establishes the standards that we must meet to be accredited. Qualitative evidence – Evidence that is not numerical. In the case of accreditation, this may include minutes of meetings, examples of student work, case studies, results of focus groups, records of discussions, reflections by faculty, etc. Quantitative evidence – Evidence that can be analyzed numerically. In the case of accreditation, this includes such things as numerical measures of student success in specific courses, survey results, graduation rates, statistical analyses of research studies, etc. 17 Course and Activity SLOs Frequently Asked Questions What are course or activity SLOs? Course or activity SLOs are statements of principal skills and abilities a student will possess upon successfully finishing a class or a college nonclass activity • Broader in scope than “objectives”, SLOs describe the major goals of the class – things that a student who successfully achieves the goals of the course will take away from the class for use in his/her future life • SLOs focus on measurable skills and abilities that can be directly assessed using student work in the class. Where do we state our course or activity SLOs? At CRC, we are going to list the course SLOs on course outline in SOCRATES; it is also good practice to state them on class syllabi • In the “Learning Outcomes and Objectives” section of SOCRATES you will be given a series of text boxes. You can enter one SLO or one objective in each text box. • The objectives are already in SOCRATES. Activity outcomes will typically be stated on the program review forms and may be on other program documents (e.g. planning documents). Who assesses course or activity SLOS? The selection of assessment tools and collection of assessment data for course or activity outcomes is done by the professor teaching the course or the CRC faculty or staff member leading the activity. Many tools can be used for the assessment, including: • Direct observation of student skills through discussion, projects, portfolios, role-plays, presentations, creative projects, practical exams, etc. • Summative assessment through exams, quizzes, problem sets, homework • Student self-assessment through reflective journals, surveys, discussions, informal classroom assessment techniques, etc 18 Is assessing SLOs different form determining grades for students? Assessing outcomes in classes may use many of the same tools used for determining grades. However, the analysis process is different when these tools are used for outcomes assessment. • Assessment for grades we look at the performance of a given student across all of the course outcomes to arrive at his or her grade. • Assessing outcomes, however, occurs when we look at the performance of all of the students in a class relative to a given outcome. How do we use the results of our assessment of SLOs? The results of our outcomes assessment for the basis of dialogue about teaching and learning with our colleagues. • This dialogue may be informal discussion between colleagues • It may involve more formal processes such as workshops, departmental meetings, etc. • Analysis of outcomes assessment data is used to make changes that support student learning. • Changes based on SLO analyses could include many things, such as requests for resources, curriculum actions, budgetary changes, modifying the mentoring of new staff and adjunct faculty, implementing new teaching strategies, etc. • The results of our outcomes assessments inform our planning and resource requests. Do we have to report the results of SLO assessments? If so, how do we do that? We will need to periodically report the results of our assessments for accreditation purposes. The Outcomes Assessment Task Force is currently working to make recommendations concerning the reporting of outcomes assessment for courses and activities. More information will be forthcoming in the next semesters. The charge of this Task Force is to: • develop a proposal that outlines the strategies and tools we could use to document the assessment of student learning outcomes at the course, program, and college levels. • recommend a system that meets the accreditation guidelines and is consistent with our campus culture, supports academic freedom, is useful to faculty and staff, and is easy to implement. 19 Writing course/activity outcomes Defining the outcomes for course and activities: Each SLO briefly describes a major skill or ability that successful students will gain from the course or activity. The focus is on the skills and abilities needed in the next course in the sequence, on the job, after transfer, etc. SLO’s provide the big picture; objectives are used to describe more specific skills or details of how the work done during the course/activity achieves the SLO. Each SLO for the course or activity will be inclusive of two or more of the course objectives (which are already listed in SOCRATES for courses). Each course or activity will usually have only a few SLOs – probably between two and five. For courses, it is often very helpful to review the course objectives to see if they can be grouped cleanly under the draft SLOs. This provides the opportunity to see if an important outcome was accidentally neglected when the objectives were written. Course and activity outcomes can include college-level abilities of three kinds: cognitive (e.g. critical thinking), psychomotor (e.g. proficiency in performing motor tasks), and affective (e.g. evaluation of ethical situations and making value judgments). The number of SLOs and the exact format will vary from discipline to discipline in ways that reflect the norms of the discipline, the requirements of relevant organizations (e.g. advisory committees for occupational disciplines) and the practices of our transfer institutions. This provides the opportunity to inform external groups (e.g. UC, CSU, accrediting bodies) of the way in which our SLOs align with their priorities. Most courses will have only a few SLOs per course, however, some disciplines will have a great many SLOs because of the requirements of their disciplinary organizations. The detail included in SLOs will also vary according to disciplinary standards. Some SLOs will be very “big picture” – e.g. “students will be able to apply critical thinking techniques to philosophical problems”. Other SLOs may be somewhat more focused – e.g. “the student will be able to evaluate medical records in order to assess compliance with regulations for health care services including Medicare, Medicaid, and Managed Care.” Entering SLOs in to SOCRATES. In the Learning Outcomes and Objectives” section of SOCRATES you will be given a series of text boxes. You can enter one SLO or one objective in each text box. The objectives are already in SOCRATES. See the publications of the Curriculum Committee for formatting details. The Methods of Instruction section should align with the SLOs. This section explains what methods will be used to teach the course and how those methods tie back to the SLOs. When writing this section, be clear and specific and tie the wording directly to the language of the course outcomes. For example, saying “Guided discussion emphasizing the analysis of cultural pattern will be used to develop critical thinking skills” is much better than saying “Discussion”. Remember, though, that the details of instructional methodology (topics within lectures, instructions for group work, etc.) do not have to be given on the course outline; these are up to each professor teaching the course. Assessment of course outcomes is reflected in two sections of the course outline: Evaluation Methods and Typical Homework Assignments. When writing these sections of the course outline 20 indicate what general types of assessments will be used to evaluate student work. Be sure to list a variety of measures that may be used in the course. This is more than just a list of types of assessment. Use complete sentences and make the connection to the outcomes quite obvious, even repeating the language of an outcome if needed. For example, saying “Students will write essays comparing and contrasting the basic components of the major psychological perspectives” is much better than saying “Assigned essays”. The connection of the assessment to the SLOs should be obvious. Remember, though that details of assessment (e.g. rubrics for grading, specific assignments, etc.) do not have to be given on the course outline; deciding these is the prerogative of each professor teaching the course. Example of outcomes and objectives from the syllabus for Bio 350 Fall 2007: These outcomes and objectives are also found on the course outline, but the information about how they are to be assessed is specific to the syllabus for this section. SLO (1) Apply the principles of ecology to the analysis of populations, communities, and ecosystems. Assessed by exam questions (typically essay questions) that require the application of ecological principles to communities and ecosystems and by questions that demonstrate the understanding of ecological terminology (these may be multiple choice questions). o Correctly utilize the basic vocabulary of environmental biology. o Describe the major processes involved in ecosystem functioning. o Analyze situations involving interactions between two species. o Describe the graphs describing exponential and logistic populations growth and the factors affecting each type of population growth SLO (2) Evaluate the impacts of human activities for local and regional environmental issues. Assessed by exam questions that require analysis of environmental impacts and by the semester project. o Analyze the components of effective environmental resource management o Describe the impacts of pollution for communities, ecosystems, and human populations o Discuss global environmental issues such as global warming, the ozone hole, etc. o Provide examples of the relevance of environmental biology to personal interests and community issues. SLO (3) Use the scientific method to pose questions and understand information and interpret scientific data as applied to environmental problems. Assessed by exam questions that require the evaluation of scientific data (these may be multiple choice or essay questions) and by the reflective essays. o Assess the results of scientific investigation into biological questions. o Draw reasonable conclusions from biological data Note: The final exam in this course is cumulative and designed specifically to allow students to demonstrate the ability to meet the outcomes of the course as described above. It includes an inclass exam and a take-home assignment. 21 Assessing course/activity outcomes Activity outcomes assessment: Many of the non-course activities are important to student learning at the college. Students learn valuable skills from their interactions with student services faculty and staff, from tutoring, from their work in the library, and from participation in athletic and artistic events, student government, etc. Many of these activities do not currently employ formal assessment tools, partly because the activities are not graded. We are working to develop useful assessments for these activities. The list of assessment tools in this chapter of the handbook suggests some tools that may be useful. Student self-assessment tools and the direct observation of student work may be particularly useful for the assessment of activity outcomes. As we develop outcomes assessment for non-class activities the focus is on assessing measurable skills and developing dialogue and analyses that will allow us to improve student learning. The results of outcomes assessment are used to identify “best practices” for the college faculty or staff who run the activities. The goal is to identify those ways of interacting with students that produce the most effective learning environment, be that in a counselor’s office, a tutoring center, or on the baseball field. When we change our practices because of what we understand about our student’s learning we “close the assessment loop”. For activities might change the methods used to interact with students, update materials used in the activity, change the mentoring of new faculty and staff, seek out professional development activities, develop new coaching/teaching/tutoring methods, etc. It is indications of these changes that the accrediting team will be looking for when they are here. The faculty and staff who lead the activities develop activity outcomes. While course outcomes will be listed on the course outlines (see below), activity outcomes will be stated on program documents such as the program review (PrOF) forms. They may also be listed on internal departmental documents as determined by the departments involved with the activities. Course outcome assessment: When we teach, we are helping our students achieve the outcomes of the course. When we give a graded assignment or test, we are measuring student learning and their success at reaching course outcomes. When we assign a grade to a student, we are really providing a single-time summary of how well he or she has achieved the course outcomes. In other words, we look at the performance of a given student across all of the course outcomes to arrive at his or her grade. Assessing outcomes, however, occurs when we look at the performance of all of the students in a class relative to a given outcome. For example, if a test includes a set of items pertaining to a course outcome, an item analysis of this set of items can tell us how our students as a whole are doing with respect to that outcome. Similarly, if a rubric for an assignment has one or more areas that correspond to a course outcome, analyzing the overall scores of students in these areas can tell us how our students as a whole are doing with respect to that outcome. The diagram 22 below illustrates this for a course that has four outcomes and assesses student learning with homework, three tests, a presentation, a course project, direct observation of students during group activities, and a student self-assessment. Outcome 1 Course project and Items from test #1, and #2 Outcome 2 Presentation and items from test #2 and #3 Outcome 3 Items from Test #2, student selfassessment and course project Outcome 4 Direct observations, presentations and homework Student 1 Grade A Student 2 Student 3 D B Student 4 C Outcome High success assessment Hurray! Results of consider using outcomes the same assessment methods for other outcomes. Low success Revise teaching methods for this outcome Moderate success Keep what worked, but consider changes to increase success High success Hurray! consider using the same methods for other outcomes. The results of outcomes assessment in our courses are used to improve how we teach. We note what worked well and celebrate that our students are succeeding on some outcomes. Then we work on improving student success on those outcomes on which they are struggling. When we change our teaching methods because of what we understand about our students learning, we “close the assessment loop”. We might change our teaching methods, update our course outline, change the course prerequisites, rewrite our assignments or rubrics, modify our handouts, choose a new text, add more group work, etc. It is indications of these changes that the accrediting team will be looking for when they are here. Obviously, the summary statement of a student’s achievement of course outcomes is his/her grade at the end of the course. The overall grade in the course will directly reflect an assessment of an individual student’s global achievement across the set of course SLO’s. However, a single overall grade doesn’t provide much information about aggregate student success on individual course outcomes. At CRC, we already use a number of measures that can help us assess student achievement on individual outcomes: • Grades on exams that measure a specific subset of SLOs, particularly if this includes an analysis of specific exam items that relate to a given SLOs. • Problem-based class projects, especially those that reflect what the student will need to do on the job or following transfer. • Portfolios reflecting skills and knowledge acquisition 23 • • • Direct observation of performance or psychomotor skills (e.g. oral communication, theater performance, athletics, lab skills, etc. ) Ungraded classroom assessments of the type developed by Cross and Angelo Student surveys and discussions Class embedded assessment of course outcomes for grading purposes Assessment tools (examples) Developed and Results shared and reported reviewed by by o Exams and quizzes Professor (may o Provide grades to students o Projects, papers, etc include shared during course. o Presentations o Grade reports at the end rubrics or o Skills demonstrations of the semester by the departmental o Assignments professor. exams). Class-embedded assessment of course outcomes for SLO analyses Assessment tools (examples) Developed and Results shared and reported reviewed by by o Exams and quizzes Professor (may o Review results as part of o Projects, papers, etc include shared the dialogue within the o Presentations rubrics or program or department. o Skills demonstrations o Brief reports filed by the departmental o Assignments department or program. exams). o Student self-assessment o Dialogue with colleagues o Informal classroom assessment Used to o o Assign grades Improve teaching methods Used to o o o o o o Assess course outcomes Improve teaching methods Review and revise curriculum Inform unit planning & PrOF Support resource requests Inform departmental dialogue Outcomes assessment tools for course or activity outcomes Direct observation of student skills through, projects, discussions, portfolios, role plays, presentations, etc.: This category of assessment tools includes a wide variety of “active” work by students ranging from classroom presentations to the production of portfolios. They provide direct measures of student skills and can involve “authentic” problems similar to those done in the professional pursuit of the discipline. They may have reliability and/or validity problems unless good rubrics are used since evaluation of student work of some types may have a strong subjective component. These tools work well for assessment of psychomotor skills (e.g. use of lab equipment, ability to fill out financial aid forms, music performance, public speaking abilities, etc.). Some examples of these tools are: • Role plays and case studies: These are the best way to assess some affective or psychomotor outcomes because they are direct measure of the skill. However, reliability problems may interfere with the use of these techniques for outcomes assessment. Detailed rubrics, checklists, and rating forms, are helpful in solving this problem. • Guided discussion: Direct observation of student response during guided discussion individually or in groups can make evident both thinking skills and foundation knowledge. This tool can also be useful for the assessment of affective skills of importance to student services such as the student’s self-efficacy in his/her ability to plan for future goals. Guided 24 discussion is relatively difficult to grade or to use to produce quantitative outcomes assessments. It can be used to provide a reflective qualitative assessment of some outcomes if rubrics are used to define levels of success for the group. • Performances, structured practical assessments, and psychomotor skill demonstrations: These effectively assess skills which are usually psychomotor in nature (e.g. lab equipment use, dance steps, construction techniques, ability to fill out a financial aid form). Validity may be increased by specifying criteria on a rubric, providing clear instructions for the tasks, and allowing a task to be performed multiple times. • Portfolios: Collection of student work in portfolios is an important student centered assessment in the arts, and has been used in many disciplines. Portfolios may have trouble with reliability and validity in some disciplines. In this case, it is helpful to (1) have a clear evaluation system and rubrics (2) have students do more than just "turn in" items, e.g. have them explain and evaluate the importance of each item in the portfolio. • Projects, reports, papers, etc.: The assessment tools in this category include research papers, lab reports, inquiry projects, creative projects, term papers, etc. These are common tools in evaluating student learning in many disciplines and, thus, there are many models to follow. Because they typically represent a major amount of work, these assessment tools can reflect more than one skill – in that case, a rubric may be needed to map outcomes to work product. Exams and Homework Questions: This category of tools includes (1) exams and quizzes ranging from blue-book essay exams, to multiple choice objective exams, to oral exams and (2) homework problem sets that are composed of items similar to those found on exams. These are very familiar methods to professors as they are very often used to conduct assessment designed to evaluate individual students in order to assign grades. They can also be used to assess individual outcomes across the class. Exam or homework questions can be linked directly to outcomes for assessment purposes. Each question can be linked to a specific outcome or objective and, thus, an item analysis can provide an outcome assessment. In a course, mapping of content to outcomes may allow each exam to focus on one or a few outcomes. Alternatively, the final exam or final homework set can be used as a summative assessment of all course outcomes. • Essays (blue book exams, homework essays, etc.): The assessment of student essays can measure outcomes related directly to writing skills but can also be linked to outcomes reflecting critical thinking, recall, analysis, evaluation, and synthesis skills. Long essays are time consuming to grade, and, since marking may be subjective; clear rubrics are very important. • Short-answer (including calculation) questions or problem sets: Short answers to specific prompts or problems can be used to assess some writing skills and to allow students to demonstrate problem solving and concept knowledge. They are moderately easy to grade and not too time consuming to write. Rubrics are helpful, especially when there is a need to be very clear about what alternatives of the answer are acceptable. 25 • Objective questions (multiple choice, true/false, matching): These provide a high reliability of scoring and are easy to grade, but are time consuming to write well. They are probably most useful for allowing students to demonstrate content knowledge and/or an understanding of terminology. They can be used to measure critical thinking or problemsolving abilities but this requires considerable work on the part of the professor writing the exam. Context-dependent questions often help with this. • Oral exams: Although rarely used, oral exams can be valid and reliable measures of some types of outcomes. This tool is most often used to assess outcomes related to interpersonal skills (e.g. interview skills) and speaking skills (e.g. foreign language classes). If questions are unrestricted, the reliability can be low and good rubrics and checklists are very important. Student self-assessments: This category includes a variety of tools that ask students to assess how well they can perform the skills and abilities learned in the activity or course. It also includes measures of student self-confidence regarding their ability to use those skills. They can be graded or can be used to help generate a rubric for future assessments and to provide feedback or outcomes assessment without producing grades. These methods can be fairly quick and easy to implement or include more extensive methods. Validity and reliability can be major problems with student self-assessments unless students are trained in how to assess their own success. • Quick classroom assessment techniques such as short surveys immediately following an activity: These tools include many quick-feedback techniques such as the one minute paper, think-pair-share, etc. They allow for modifications of teaching strategies in a rapid turnaround basis. While good for formative assessment, these tools may not be as useful for summative assessment. • Student journals or reflective essays: These provide rich information on student progress and allow the assessment of affective outcomes in a detailed way. Only a few disciplines (e.g. ECE) have a tradition of using these tools. • Surveys conducted at the completion of a course, activity, or program. These can be used to ask students about their confidence in their ability to successfully conduct the task learned in the course, activity, or program. They are useful for summative assessment, particularly of program outcomes. Long surveys are time intensive and require substantial resources to implement. • Student self-grading of an assignment or project: Students benefit from learning how to assess their progress and can apply this to future work. Validity can be major problems with student self-assessments unless students are trained in how to assess their own success. • Individual discussions between a student and a teacher: One-on-one discussions provide a rich source of affective information. It requires substantial planning to assess outcomes using these tools. 26 Using the results of course and activity outcomes assessment: dialogue, change, and planning We analyze student success on course and activity outcomes so that we can utilize that information to identify what works well and improve what doesn’t. When the data indicates the need, we can improve our students’ learning by modifying: • Materials provided to students • Mentoring of new faculty and staff • Course assignments • Methods used to assess student learning • Readings and texts assigned in courses • Methods used in lecture • Methods used in laboratory classes • Methods used in non-class activities • Instructions for student work Dialogue, dissemination, and documentation for course and activity outcomes assessment The OATF has developed a formal process for reporting the assessment of those outcomes aligned with the most important issues affecting student success in each program. (see page 11). As part of this process, each program will be using brief forms to report the assessment results of selected program, course, or activity outcomes each year. . Samples of the OATF forms can be found in the appendix to this handbook. Processes for reporting assessments have been developed by the Outcomes Assessment Taskforce. The key part of the process is that program faculty and staff identifies the most important learning issues affecting the success of their students and focuses their assessment reporting efforts on outcomes related to those issues. Brief reporting forms will be collected each semester. The forms will be collected by the OATF. Samples of the OATF forms can be found in the appendix to this handbook. The information will be collated for use in program review, planning processes, and collegial dialogue. In Fall 2008, we will implement these processes for the first time on a college-wide basis. The process consists of two stages in each academic year: Fall Semester: At the beginning of the fall semester programs will be asked to identify the most important learning issues facing their students and the outcomes that align with those issues. More than one issue may be identified. Programs will focus their outcomes assessment reporting for the year on those outcomes. These may be college-wide, GE, program, course, or activity outcomes. Each program will complete a brief form that describes the following: • the important issues identified by the program faculty and staff • the outcomes associated with those issues • • a summary of plans to assess, and report the assessment of, those outcomes 27 Spring Semester: In the spring programs will report on their assessments of the outcomes identified in the fall. They will complete a brief form that reports the following • the important issues and related outcomes identified in the fall • the methods used to assess those outcomes • the results of the assessments • what was learned from that assessment; i.e. what was identified as effective and what can be changed to improve student success based on the fall assessment We can also learn from one another and conduct dialogue about student success on our learning outcomes in many less formal ways. There are a number of ways, other than the OATF forms, in which we can record and/or disseminate our efforts. o During department meetings, we can set aside time to share with one another methods that have helped our students and gather advice on ways to improve where needed. o Modifications to materials provided to students (e.g. syllabi, student services materials, etc.) will show the changes in our classes over time. For course outcomes, if syllabi are kept on file in a location open to all professors, this would result in a record that we could access to get good ideas. o Revisions of course outlines will show the evolution of courses over time. Archived versions of the course outline in SOCRATES or in department files result in a record of these changes. o Workshops offered through the CASSL, and other staff development activities, could be set up so that we can share the wisdom we have gained over our semesters of teaching experience. o Various sections of courses in some department share common grading rubrics for some tasks. Similarly, the outcomes of some activities conducted by numerous faculty or staff (e.g. counseling sessions, tutoring, etc.) can be assessed using common rubrics. The professors conducting those courses/activities could meet to examine student success on these tasks and brainstorm on what methods could improve class outcomes. o In the self-study we complete as part of our evaluation process, we could reflect on what the data shows as the strengths of our methods at encouraging student success, on our growth as teachers, and on the new techniques that we have employed to improve our students’ learning. 28 CHECK LIST FOR COURSE SLO DEVELOPMENT Writing Outcomes: Is the focus on the big picture – a fundamental result of the course? Do you describe what the student can DO with the knowledge they have acquired in the class? Can the outcome be assessed based on work produced by the student? Do the SLOs represent skills and abilities that the students need to carry with them to the next stages of their lives and/or education? Is the language clear? Designing Assessments: Do the assessments clearly link to the SLOs? Are multiple measures for assessment listed? Is it clear that the results of the assessments will be used to improve the course? Aligning course elements: Do the Methods of Instruction, Methods of Evaluation, and Independent Assignments sections of the course outline clearly link to the SLOs? Is enough information provided in these sections so that it can guide the teaching of a course or the writing of a syllabus? Is there enough flexibility for individual professors teaching the course? 29 Program SLOs Frequently Asked Questions What are program SLOs? Program outcomes reflect cumulative learning throughout an entire program of study. • The “program” could be the set of courses leading to a degree or certificate. However, there may be programs that do not lead directly to degrees or certificates. • Program outcomes answer the question “What will they be able to do out there based on what they have learned here?” • Program outcomes define the major skills and abilities that a student completing a program will take with him or her after leaving CRC. • Program outcomes are defined by the faculty and staff of a program. Where do we list our program SLOs? Program outcomes are stated, and the results of outcome assessment are discussed, on the program review documents (PrOF). • Assessment of program outcomes should be clearly reflected in our planning and resource requests as given in PrOF. • The PrOF forms are currently being revised; more information will be available from the Planning Committee. • It should be clear how the activities within the program support specific program SLOs. This can be done by filling out a simple “mapping” grid • All program outcomes should be supported by one or more courses or activities How is the assessment of program SLOs done? Depending on the structure of the program, the assessment of program outcomes may occur in a variety of ways. 30 • • • The “normal” assessments done in courses or activities across program, e.g. using the same tools used for the assessment of course or activity outcomes that map to the program outcomes, Summative assessments in “capstone” courses or activities such as capstone projects, portfolios, exams, etc. Assessments of those students who have completed the program, including surveys of current program completers, follow up studies of alumni, etc. How do we use our assessments of program SLOs? Assessment of program outcomes is used to promote dialogue, improve the program courses/activities, and support planning and resource requests from the program. In response to outcome assessment, we may change: • Intra-departmental processes such as budget allocation, mentoring, teaching strategies, program offerings, etc. • Curriculum actions, including program options, new courses, requisites for courses, program descriptions, course or activity format (e.g. on ground versus on line), etc. • Requests for resources including equipment, faculty, staff, or tutoring positions, facilities, outreach resources, staff development resources, etc. Do we have to report the results of SLO assessments? If so, how do we do that? We will need to periodically report the results of our assessments for accreditation purposes. The Outcomes Assessment Task Force is currently working to make recommendations concerning the reporting of outcomes assessment for courses and activities. More information will be forthcoming in the next semesters. The charge of this Task Force is to: • develop a proposal that outlines the strategies and tools we could use to document the assessment of student learning outcomes at the course, program, and college levels. • recommend a system that meets the accreditation guidelines and is consistent with our campus culture, supports academic freedom, is useful to faculty and staff, and is easy to implement. 31 Introduction: Defining programs Functional definition of programs: Historically, we have identified programs either by looking at a list of degrees and certificates or by grouping related academic disciplines. We are moving to a more functional identification of programs. At CRC, a collegiate “program” is a group of related courses or other activities that align with a cohesive set of student learning outcomes. It may help to begin by thinking about this question: What are the roles of your program in the broader context of the college? Here, “roles” are defined broadly as the functions of the program with respect to the students and the college. With respect to the students the question is “why do students participate in your program?” Why do they take your classes? Why do they participate in other activities in the program? Finally the most important question: can you write a single set of outcomes that apply to all students in the program? In order to make this process more flexible for the many things we do at CRC, three types of programs are identified. [A list of programs is available from the college Research Office]. Instructional Programs: These are programs in which the major interaction between students and CRC faculty/staff occurs in credit-granting courses. Examples include programs from Business to Social Science and departments from Accounting to Work Experience. In the catalogue, these are usually called “Areas of Study”. Non-instructional Programs - Student Services/Student Support Programs: These are programs in which CRC faculty/staff work directly with students in a variety of settings other than credit granting classes. The catalogue defines student support services as those which “…assist students with career decision-making, problem resolution, and goal achievement” (catalog p.9). This includes a wide range of services such as Assessment Services, Counseling, DSPS, EOPS, CARE, Heath Services, Library, Math Center, Reading Writing Center, MESA/CCP, Student Development, Tutoring, etc. These programs may have Service Area Outcomes instead of, or in addition to, Student Learning Outcomes. Non-instructional Programs - Instructional Support/Administrative Services Programs: These are program that indirectly support teaching and learning. They include Admissions and Records, Bookstore, Business Services, CASSL, Research Office, Custodial Services, Instruction Office, etc. These programs may have Service Area Outcomes instead of, or in addition to, Student Learning Outcomes. Notes on instructional programs: Degrees or certificates typically relate to a cohesive set of outcomes, and so we will retain many of our traditionally identified programs. However, some departments may want to revisit how their programs are defined. For example, an English Department may decide that they are two programs: Developmental English and Transfer English, or they may decide that they are all one English Department. 32 There may be instructional programs that do not lead directly to degrees or certificates. For example, there is no current CRC degree or certificate in “History”. Nonetheless, there is a group of courses that could clearly align with a set of student learning outcomes. Alternatively, the History and Political Science departments may find that they support a single cohesive set of outcomes and so could be considered together as one program. Similarly, “Counseling” is clearly a program with respect to being able to identify student outcomes, but is not associated with any particular degree or certificate. One instructional program may include more than one degree or certificate. For example, there are three Biology Degrees available at CRC and students take courses in Biology for several reasons: to transfer to a four-year school in as a biology major, to prepare for the eventual goal of attending a professional school in medicine, to work toward more immediate employment in health care (e.g. as LVNs), or to fulfill GE requirements. Nonetheless, the Biology Department has identified a single set of outcomes that applies to all students in Biology classes and “Biology” has identified as only one program. Program structure: After a program has been defined in general, it is useful to think about the overall structure of the program in terms of the course and activity sequences. Programs, or parts of programs, can be thought of as having one of three structures: horizontal, vertical, or mixed. This is relevant to how program outcomes are reflected in the course outcomes. Horizontal program structures are those in which students may participate in courses or activities in any order. In some cases, students typically take only one or two courses in the discipline, although many more courses may be offered. There is not prerequisite sequence that steps students through the courses in a specified sequence. In the most extreme example, if students take only one of the courses or activities in the program, then whichever one they take will have to move them to achievement of all of the program outcomes. In this case, program outcomes and the outcomes of each course/activity may be essentially the same. If student have to take either courses A+B or courses C+D, then we would expect that those combination of courses include all of the course outcomes. It is seldom that simple, but may be in some cases. The GE component of the college-wide outcomes works in this horizontal fashion. Students are required to take GE courses in several areas, but in no particular sequence. GE option 1 GE option 2 GE option 3 GE option 4 Some non-instructional programs may have a similarly horizontal structure, where a student is required to participate in all of the activities, but chooses the various activities in any order: Activity A Activity B Activity C Activity D 33 In this case, we would expect that sum of all of the course/activity outcomes will contain all of the program outcomes. Vertical program structures are those in which there is a linear sequence of courses or activities, which students move through in a specified order. A sequence of courses may be defined by prerequisites. For example, Math, English, and Foreign Languages contain some linear sequences of courses. Vertical structures may also be based on students moving through a set of non-course activities in a specified sequence. Capstone Prereq 3 Prereq 2 Prereq 1 In this case, the “capstone” course or activity in the sequence may capture all of the program outcomes, or each of the prerequisite courses may incorporate some of the outcomes of the program. For example, in a tightly linked vertical program, each course or activity may have the same general outcomes, with the student working at progressively more advanced levels as the student moves through the sequence. In this case, the capstone course or activity may include a summative assessment of all of the program outcomes. Alternatively, each course or activity in the sequence may develop very different skills and abilities. In this case, each course or activity would align with a subset of the program outcomes. It is more likely, in this example, that each course or activity would assess the subset of programs outcomes that it covers. Mixed program structures contain some vertical components and some horizontal components. Many of our degree and certificate programs work this way. For example, students may take one or two introductory courses defined by prerequisites and then have a broad choice of more advance courses that can be taken in any order. Such a mixed structure can be quite complex and may involve courses from more than one discipline (for example, Math or Chemistry as part of a Biology degree). Mapping courses to program outcomes may also be complex for these programs. Disc. 2 Disc. 2 Core Core Core Disc. 2 Disc. 3 Disc. 3 Disc. 3 Elect Elect Req. 1 34 In such complex situations, a map showing which program outcomes are supported by which course is essential. Our program review (PrOF) documents include such mapping. A hypothetical example is shown below: Outcomes for a hypothetical “Water Art” Program M = major component, m= minor component Course 1: Underwater basket-weaving Course 2: Synchronized swimming theory Course 3: Pool and fountain design Course 4: Advanced water arts studio Activity 1: Water tasting sensory experience Activity 2: Synchronized swim meets Activity 3: Water theatre performance Art Design Taste Swim M M m M m m M M M M m m m M M M m 35 Writing program outcomes Defining program outcomes Program SLOs answer the question “what will they be able to do out there based on what they have learned here?” SLOs describe the major outcomes – things that a student who successfully completes the program will take away for use in his/her future life. They may include three kinds of abilities: cognitive (critical thinking), psychomotor (proficiently conducting specific motor tasks), and affective (evaluating ethical situations and making value judgments). Before developing specific SLOs for the program, it is helpful to define the overarching expectations and goals of the program for its students. Another way to think about this is to ask about the major results of the program if is working in an ideal fashion. What are the expectations of the broader world “out there”? What do the “receivers” of our student expect? If the students are leaving here for immediate employment, what skills are they expected to arrive with on the first day of the job? If students are transferring from here to a four-year school, what skills will they need in their classes after transfer? Some external institutions (e.g. IMPAC, transfer schools, accrediting agencies, advisory groups, etc.) may provide direct information on the skills and/or knowledge expected from our students. These should definitely be considered when writing program SLOs. For example, if many of the students in a CRC program transfer to CSUS, the CRC program SLOs should make it clear that our transfer students are prepared to succeed at CSUS. This is a chance to strengthen our relations with transfer institutions and employers. It is also useful to consider the elements that are shared across many of the courses/activities within the program. • Common content themes: Many of the courses or activities within a program will reflect the same broad content themes or ideas. For example, many Biology classes may include content related to the functioning of cells, many Art courses may include content related to the importance of color and line, many Library activities may include content related to use of online information sources, etc. These common themes suggest areas that are of broad and significant importance for student learning. • General skills: Many of the courses or activities within a program may share an emphasis on the development of related skills, perhaps at increasingly sophisticated levels. These common skill sets suggest areas that are of broad and significant importance for student learning. • Typical assessment tasks: It is useful to compare and contrast the typical assessment tasks within the program. In some cases, it may be important that similar tasks and scoring rubrics are used in sequential courses. In other cases, it may be important that different activities or courses develop the students’ ability to respond to multiple assessment measures. It should be clear how the activities within the program link to specific program SLOs. This can be done, perhaps within the program review, by a narrative description, and/or by a grid showing the program SLOs on one axis and the aligned activities on the other. Some departments at CRC are developing models for doing this. If it is obvious which courses support specific program outcomes, then the link between course performance and program outcome will be 36 straightforward. For example, if there is a program SLO in a PE degree program concerning a psychomotor skill it should be clear which course(s) develops that skill. If a CIS certificate program has an SLO concerning knowledge of a specific operating system, it should be obvious which course(s) develops that knowledge. “Mapping” is a term that refers to showing how courses or activities relate to specific outcomes. For example, the courses in a program can be “mapped” to the program outcomes. This would show which courses support specific program outcomes. One relatively easy way to do this is with a simple table, as shown below: Outcomes for a hypothetical “Water Recreation” Program M = major component, m= minor component Course 1: Intermediate Swimming Course 2: Synchronized swimming theory Course 3: Water parks design Course 4: Canoeing and Kayaking Activity 1: Surfing field trip Activity 2: Synchronized swim meets Activity 3: Water theatre performance (capstone) Art Team Ind. Swim skills skills M M m M M M M M M M M M M m m M M 37 Assessing program outcomes Assessment tools Course-embedded assessments: For many CRC programs courses are the main activity in which students participate as part of the program and, thus, an analysis of student performance in courses will form the major part of assessing students’ achievement of program SLOs. Many of the tools used for assessing courses and activities can be used to assess those program outcomes that map to the course or activity. See the list of course/activity assessment tools in the previous chapter. It should be noted that while the same data may be used, analysis for program outcomes isn’t exactly the same as that used in assessing course outcomes. While the data in program assessment may be based on performance in courses, the context of the analysis is the program, not a single course. In a program assessment, course performance data may be used to ascertain in which courses students achieve the greatest success, to identify gatekeeper courses, to validate course requisites and course sequences, to analyze the placement of disciplinary content into specific courses, or to develop guidelines for capstone projects or portfolios. Information about program completers: The examination of student performance in courses will not provide full assessment measures for all SLOs. This likely because (1) Some program outcomes will be too broad to be measured in one course or activity, but are supported by a variety of activities and will need to be measured broadly and (2) Some CRC programs do not have courses as the main activity within the program. For example, Counseling is clearly a program that impacts students through activities other than just courses. Again, it should be clear how the activities of the program support program outcomes, but measurement will plainly not involve looking at performance in courses as the major measure of outcome achievement. It may be necessary to devise new assessments for these SLOs, which could be part of program review. These might include both direct and indirect measures such as: • Evaluation of capstone project or portfolios • Specific assessments built into end-of-program courses • Reports from employers • Pass rates on licensing or certification exams. • Follow up studies of students after transfer • Exit surveys or interviews of student near or at completion of the program • Analyses of student success based on direct observations of students by the faculty who teach those courses that occur near the end of a program. • Evaluations of students in internships or work experience courses related to the program. • Interviews or surveys of instructors, program coordinators, tutors, and others that work with students completing the program. Some of this data can be easily gathered, other data cannot. Much of this is new to our processes. It will take some time to adjust to gathering data on global/cumulative outcomes that cannot be assessed by an analysis of student performance in individual courses. Fortunately, not all, perhaps not many, program SLOs will be of this nature. 38 Measures of student participation in activities on campus and use of campus resources: These may be useful to assess student service outcomes and may provide direct assessment of some skills (e.g. ability to fill out financial aid forms, ability to utilize campus resources, etc.). They can also be indirect measures of student skills. For example, measures of student activity/participation on campus may provide indirect measures of college-wide or program outcomes related to student interaction with the college. Program completion rates: These are useful measures of student success in some ways, but are only indirect measures of student success in specific program SLOs. Such information is more useful if it provides a greater level of detail than simply an aggregate measure for all students in a program. Such data can be more useful if information concerning program completion also provides data on why students do or do not complete the program, or which groups of students are more/less likely to complete the program. For example, course success data might provide information on the “gatekeeper” courses that are the most limiting factor in progress toward program completion. In another example, we may be able to begin to understand which students fail to complete programs if we break down the data by demographic variable (e.g. economic status, ethnic group, etc.). 39 Using the results of program outcomes assessment: dialogue, change, and planning Dialogue, dissemination, and documentation for program assessment Major changes to a program are seldom undertaken without substantial dialogue among program faculty and staff. However, assessment of program outcomes provides for an ongoing dialogue, rather than periodic meetings that are held only when a major decision is to be made. The assessment of program outcomes can be used to answer many questions leading to improvements in student success. For example, in programs that have sequential courses, comparisons of the progress of students through the sequence will provide an overall view of where “gatekeeper” courses occur. A closer analysis can be undertaken, perhaps as part of a program review, to identify the SLOs that act as “keys” to the gates and the practices that support student success in those key outcomes. Evidence of our dialogue with respect to outcomes development and assessment might come from a variety of sources. For example, records of data analysis, records of department or committee meetings, email strings from the department, examples of changes to program structure, examples of professional development sought out and attended, etc. Closing the loop by making changes: Assessment of program outcomes allow us to “close the loop” by revising the way the program functions. The program can respond to the information from outcomes assessment in many ways. Responses to program outcomes assessment can include (but are certainly not limited to): Intra-departmental changes ° Changing how budget is allocated within the program ° Modifying mentoring of new staff and adjunct faculty ° Implementing new teaching strategies ° Expanding/decreasing offerings in some parts of the program Curriculum actions: ° Developing new programs or new program options ° Developing new courses ° Changing the requisites for courses ° Modifying program descriptions ° Developing suggested sequences of courses within programs ° Moving courses to new formats such as changing lecture/lab units, moving a course to a distance education format, etc. ° Revising course outlines (and syllabi) with respect to course content, topics, methods of instruction, or methods of evaluation. Requests for resources ° Purchasing new equipment or technology ° Adding faculty or classified staff 40 ° ° ° ° ° Increasing tutoring opportunities within the program Updating of facilities or develop new facilities Increasing outreach both externally and within CRC Increasing staff development activities Developing more effective links to services to students, library activities, matriculation activities, etc. Reporting program outcomes assessments and responses: Annual reporting process: The OATF has developed a formal process for reporting the assessment of those outcomes aligned with the most important issues affecting student success in each program. (see page 11). As part of this process, each program will be reporting the assessment results of selected program, course, or activity outcomes each year. Samples of the OATF forms can be found in the appendix to this handbook. Processes for reporting assessments have been developed by the Outcomes Assessment Taskforce. The key part of the process is that program faculty and staff identifies the most important learning issues affecting the success of their students and focuses their assessment reporting efforts on outcomes related to those issues. The information will be collated for use in program review, planning processes, and collegial dialogue. In Fall 2008, we will implement these processes for the first time on a college-wide basis. The process consists of two stages in each academic year: Fall Semester: At the beginning of the fall semester programs will be asked to identify the most important learning issues facing their students and the outcomes that align with those issues. More than one issue may be identified. Programs will focus their outcomes assessment reporting for the year on those outcomes. These may be college-wide, GE, program, course, or activity outcomes. Each program will complete a brief form that describes the following: • the important issues identified by the program faculty and staff • the outcomes associated with those issues • • a summary of plans to assess, and report the assessment of, those outcomes Spring Semester: In the spring programs will report on their assessments of the outcomes identified in the fall. They will complete a brief form that reports the following • the important issues and related outcomes identified in the fall • the methods used to assess those outcomes • the results of the assessments • what was learned from that assessment; i.e. what was identified as effective and what can be changed to improve student success based on the fall assessment Reporting on the PrOF forms: A multi-year summary of the changes implemented in response to program SLO assessment will be part of the program review process. The PrOF forms are currently being revised and the Outcomes Assessment Taskforce is working to develop recommendations for documenting outcomes. Look for more information from the Planning Committee in the near future. 41 General college-wide outcomes and GE SLOs Frequently Asked Questions When did we write the CRC college-wide outcomes? General college-wide outcomes for CRC were approved by the Academic Senate in Spring 2005. • The general college-wide outcomes were developed by faculty participating in focus groups, meetings, email discussions and other processes led by CASSL. • The general college-wide outcomes were submitted to, and approved by, the Academic Senate during the 2004-2005 academic year. • The general college-wide outcomes are considered a living document that will undergo periodic review and revision. What areas do the CRC college-wide outcomes address? The college-wide outcomes are related to five broad areas, each of which has an overarching goal and specified outcomes. • Graduation competencies • General education (these are further developed and expanded by the CRC GE outcomes) • Program completion • Student development as self-reliant learners • Society, ethics, and pluralism Who developed the GE outcomes? GE outcomes, specific to each GE area, were developed by faculty in a process led by the CRC Curriculum Committee. 42 • • These specify the major skills and abilities that a student will gain through completion of the general education curriculum at Cosumnes River College. The Curriculum Committee will work to ensure that courses approved to meet a GE area align with the appropriate GE outcomes How will we assess college-wide and GE outcomes? Assessment of the general college-wide outcomes and GE outcomes will encompass a broad range of assessment tools. • Both institutional measures (including research office data and student services analyses) and course-embedded assessments can be used. • The Outcomes Assessment Taskforce is currently developing recommendations for the assessment of general college-wide and GE outcomes. • Assessment of college-wide outcomes may be coordinated by the college Research Office. How will we use the results of the assessment of college-wide and GE outcomes? The results of outcomes assessment at the college level will allow us to improve the overall success of students at CRC. • Results will be compiled and disseminated by the Research Office and CASSL. • Faculty dialogue about the results of the assessment of college-wide and GE outcomes will occur in departments, through the Academic Senate, and through CASSL. 43 Current General College-wide outcomes Approved by the CRC Academic Senate, May 2005 Note: This is a living document and will be reviewed, and perhaps modified, on an annual basis. Area 1: College Outcomes related to graduation competencies Learning Expectation related to graduation competencies: Students will experience an academically rigorous learning environment that challenges them to develop the critical verbal, written, and quantitative skills needed to analyze complex issues. College Outcomes: As the result of a variety of learning experiences, students will be able to: a. Use quantitative reasoning (e.g. mathematics) to solve problems. b. Express ideas and facts clearly and completely in a variety of written formats. c. Comprehend and analyze written material and evaluate arguments from reading material. d. Speak and listen critically and recognize the value of a wide range of communication styles. e. Use the English language proficiently. Area 2: College Outcomes related to general education Learning expectations related to general education: Students completing any of the GE patterns will develop an understanding of the basic concepts and major modes of inquiry used in a variety of disciplines. College Outcome: As the result of a variety of learning experiences, students will be able to: a. Articulate an awareness of a variety of disciplinary perspectives and the relevance of these perspectives to one’s own life. b. Correctly utilize the basic vocabulary of several disciplines. c. Utilize, at a basic level, the tools of inquiry and information acquisition of several disciplines. d. Demonstrate the intellectual skills, creative capabilities, values, and attitudes that will make them effective learners and citizens. e. Demonstrate an awareness of the connections between disciplines. Note: These outcomes are further developed and expanded by the CRC GE Program Outcomes Area 3: College Outcomes related to program completion Learning expectation related to program completion: Students completing a degree, disciplinary transfer program, or certificate will develop a depth of understanding, including critical cognitive, psychomotor and affective skills, in at least one discipline (transfer or occupational). College Outcomes Area 3: As the result of a variety of learning experiences, students will be able to: a. Use of the modes of analysis and critical thinking in the major discipline of study * as applied to significant problems. 44 b. Communicate a complex understanding of content matter of the major discipline of study. c. Demonstrate competence in the skills essential to mastery of the major discipline of study. d. Use information resources to gather discipline-specific information. e. Explain the importance the major discipline of study in the broader picture of society. *“major discipline of study” = the discipline in which the student completes program Area 4: College Outcomes related to student development as self-reliant learners Learning expectation related to student development as self-reliant learners: Students will make progress toward becoming engaged and self-reliant learners demonstrating habits of intellectual inquiry and striving toward their maximum potential. College outcomes: As the result of a variety of learning experiences, students will be able to: a. Actively engage in intellectual inquiry beyond that required in order to pass classes. b. Identify personal goals and pursue those goals effectively. c. Confidently seek out the information and resources needed to develop academically and personally. d. Incorporate what is learned at school into daily life and use this information to make positive personal, educational and career choices. e. Utilize skills from the “academic tool kit” including time management, study skills, etc. f. Be actively involved in campus life and express a sense of engagement with the campus culture. Area 5: College Outcomes related to society, ethics and pluralism Learning expectation related to society, ethics and pluralism: Students will become more prepared to contribute to a diverse democratic society with a pluralistic perspective of the world. College outcomes Area 5: As the result of a variety of learning experiences, students will be able to: a. Participate in the larger community beyond campus in a positive manner demonstrating an understanding of personal responsibility in the larger context. b. Undertake thoughtful consideration of divergent points of view and utilize multiple perspectives in considering information. c. Develop a foundation for cultural pluralism, a rejection of previous personal prejudices, and knowledge of and comfort with others unlike themselves. d. Recognize the ethical dimensions of decisions and actions as well as demonstrate the ability to engage in the ethical reasoning necessary to exercise responsibility as an ethical individual, professional, local, and global citizen e. Demonstrate civility, empathy, interpersonal competence, social responsibility, and peaceful conflict resolution. 45 Current General Education Student Learning Outcomes Overarching goal: Students will experience a rich learning environment that assists them in developing a habit of intellectual inquiry and a broad perspective of the world. GENERAL SKILLS – Upon completion of the general education curriculum at Cosumnes River College the student will have achieved proficiency in all of the following general skills. I. Critical Thinking A. Students will identify important questions and points, distinguish interpretations and opinions from more factual information, and construct an accurate interpretation of the viewpoints and reasoning of a written or spoken text. B. Students will present information in an organized manner. C. Students will evaluate information or data for quality, validity and bias to determine if it is objective and reliable. II. Information Acquisition and Analysis: A. Students will access, synthesize and evaluate information using a variety of print and electronic sources including computer networks. B. Students will utilize technological resources in order to explore and express information. C. Students will determine the extent of information needed, evaluate the information and its sources critically, and ethically and legally apply gathered information to personal and community issues. III. Ethical Capacities: A. Students will apply ethical reasoning skills within the various GE areas and work toward a personal resolution of ethical issues. B. Students will show an appreciation of ethical principles as applied to personal and civic choices. C. Students will realize and apply the responsibility to use knowledge wisely. D. Students will assume civic, political, and social responsibilities locally, nationally and internationally. DISCIPLINE SPECIFIC SKILLS (Graduation Requirements): I. Natural Sciences (C1) – Upon completion of this GE requirement the student will have satisfied all of the following: A. Students will explain the core perspectives of the scientific method and apply it to at least one scientific discipline. B. Students will solve introductory problems of a conceptual and/or numerical nature of at least one scientific discipline. C. Students will accurately apply the basic vocabulary and concepts of at least one scientific discipline verbally and in writing. 46 D. II. Students will recognize the use and misuse of scientific concepts in society including politics and the media. American Institutions (C2) - Upon completion of this GE requirement the student will have satisfied all of the following: A. Students will describe both verbally and in writing the historical development of American political, social and cultural institutions including the key individuals and/or movements and their motivations. B. Students will demonstrate an understanding of the basic national, state and/or local political processes including the US Constitution and the rights and obligations of citizenship through responsible engagement in civic duties. C. Students will describe both verbally and in writing the role of diverse ethnic, religious and social groups in American political, economic and social development. III. Social and Behavioral Sciences (C3) - Upon completion of this GE requirement the student will have satisfied all of the following: A. Students will accurately apply the basic vocabulary and concepts of at least one social or behavioral science discipline verbally and in writing. B. Students will examine the possible causes and suggest solutions to introductory problems of a conceptual nature using the methods of at least one social or behavioral scientific discipline. C. Students will recognize the use and misuse of social and behavioral science concepts in society including politics and the media. IV. Humanities and Arts (C4) - Upon completion of this GE requirement the student will have satisfied all of the following: A. Students will demonstrate an appreciation of artistic endeavors, cultural expressions, ideas and/or institutions through non-empirical, analytic, interpretive studies and critical thinking projects. B. Students will articulate the development of and relationships between different civilizations, cultural traditions, ideas and/or institutions through the application of non-empirical, analytical reasoning. C. Students will demonstrate the ability to critically evaluate analyses and interpretations (including significant historical or contemporary analyses and interpretations) of arts, ideas, skills (including language), and/or institutions. D. Students will clearly express their own analyses and interpretations of arts, ideas, skills (including language), and/or institutions, and will properly use the vocabulary appropriate to the field. V. English Composition (C5a) - Upon completion of this GE requirement the student will have satisfied all of the following: A. Students will express their ideas clearly in well-organized written messages. B. Students will use correct and appropriate conventions of mechanics, usage, and style in written communication. C. Students will comprehend main ideas and reasonably interpret written information. D. Students will properly document sources of information. 47 VI. Communication (C5b) –Upon completion of this GE requirement with a communications course (COMM, MGMT), the student will have satisfied all of the following: A. Students will conduct audience analysis to design an appropriate purpose, topic, style and speech structure within formal presentations. B. Students will construct an effective presentation to a specific topic by collecting relevant information and employing credible evidence with proper documentation. C. Students will express appropriate nonverbal messages to audiences that adhere to proper conventions of delivery (e.g., eye contact, movement, gestures, vocal qualities, effective presentational aids, etc.). D. Students will listen effectively to comprehend spoken messages, analyze information critically and consider multiple perspectives. E. Students will determine and use appropriate communications technologies to convey information. VII. Analytical Thinking (C5b) - Upon completion of this GE requirement with an analytical thinking course (ACCT, CISC, CISP, ENGWR, ENGCW, HIST, JOUR, PHIL, PSYCH 335, SOC), and dependent upon the courses taken to fulfill it, the student will have satisfied at least two of the following: A. Students will organize information utilizing logical structure of inductive and/or deductive forms. B. Students will construct an accurate and/or logical interpretation of reasoning while applying a framework of analytic concepts. C. Students will analyze reasoning processes to evaluate issues, value judgments or conclusions that determine the quality, validity, and/or reliability of information. D. Students will recognize and identify inference, bias, narrowness, and/or contradictions within points of information. VIII. Quantitative Reasoning (C5b) - Upon completion of this GE requirement with a quantitative reasoning course (MATH, STAT, PSYCH 330) the student will have satisfied all of the following: A. Students will effectively organize, present, interpret and summarize quantitative information using symbolic, numerical and graphical methods. B. Students will solve problems by evaluating the available information and classifying the type of problem, choosing an appropriate technique, applying the technique accurately, and verifying whether or not the result is reasonable. C. Students will use appropriate quantitative skills including arithmetic and basic algebra to solve problems applicable to occupational and personal activities. IX. Living Skills (C6) – Upon completion of this GE requirement and dependent upon the courses taken to fulfill it, the student will have satisfied at least one of the following: A. Students will demonstrate concepts of physical and emotional wellness to make wise lifestyle choices and will develop the skills and competencies to understand themselves as whole persons (integral to their environment). 48 B. C. D. E. F. G. Students will establish daily habits for caring for their bodies including appropriate physical activities in order to maintain or improve physical and mental health and prevent illnesses. Students will identify aspects of social, physical, and/or emotional well-being for the community while considering present and future conditions in society. Students will recognize the need for healthy personal relationship and lifestyle with members of their family and community. Students will be able to understand and apply the scientific method to health, fitness and nutrition information to determine the validity of that information and in so doing, understand the relationship between scientific research and established knowledge. Students will develop skills and competencies for effective and competitive workforce performance. Students will demonstrate information competency through the ability to locate, evaluate, and effectively use needed information ethically and legally, while understanding the economic, legal, and social issues surrounding the use of information. X. Ethnic/Multicultural Studies (D) - Upon completion of this GE requirement the student will have satisfied all of the following: A. Students will become effective citizens of a diverse and multicultural society, will demonstrate thoughtful consideration of divergent points of view, and will develop a foundation for cultural democracy based on a respect for cultural diversity. B. Students will critically analyze cultural issues to recognize tendencies towards bias and stereotyping, understand the implications of bias on societal and individual health, and will stand up for themselves or others in the face of bias. C. Students will recognize, appreciate and understand intra-cultural diversity and the cultural practices of historically underrepresented groups in the United States as well as the contributions of non-Eurocentric cultures to world civilization or the United States. D. Students will demonstrate verbally and in writing an understanding of the various aspects of culture beyond race and ethnicity. E. Students will demonstrate comfortable, empathetic interaction with people from diverse backgrounds. F. Students will construct a knowledgeable and confident self-identity, and will exhibit through their actions, speech, or writing, a foundation of cultural understanding and respect. 49 Assessment of general college-wide outcomes and GE outcomes Broad institutional assessments: Some college-wide or GE outcomes are expected to accrete gradually over the course of a student’s experiences at CRC and are, thus, very difficult to assess in any one course or activity. For these outcomes, the college Research Office can be useful in developing and analyzing assessments. • Some information could be gathered by road-based surveys that gather selfassessment data from a cross section of students at the college. • Surveys of program completers (e.g. graduating students or certificate completers) may be developed which allow us to gather information through student selfassessments of their summative experience. • Additional, indirect, data could come from institutional measures such as student success rates or completion rates in courses, course sequences, or activities that support the college-wide or GE outcomes. Assessment of college-wide outcomes and GE outcomes may be mainly coordinated with the assistance of the college Research Office. The Outcomes Assessment Taskforce is currently working with the Research Office to develop methods to assess the general college wide and the GE outcomes which have been defined at CRC and ways in which we might best use the information from this assessment. Class or activity embedded assessments: Some ongoing assessments of outcomes that are part of courses or activities may also be assessments of college-wide or GE outcomes. This may occur because the outcomes of a course or activity may map directly to a college-wide or GE outcome. For example: • A class in Ethics may have outcomes that map directly to the college-wide outcome 5d that states that a student will be able to “recognize the ethical dimensions of decisions and actions as well as demonstrate the ability to engage in the ethical reasoning necessary to exercise responsibility as an ethical individual, professional, local, and global citizen”. • In Biology 307 (a GE class), one outcome states that students will be able to “utilize the scientific method and evaluate the scientific validity of information presented by the media and other sources.” The course assessments for this outcome include inquiry projects and exam/quiz questions. The course outcome maps directly to GE Outcome IA, which states that students will be able to “explain the core perspectives of the scientific method and apply it to at least one scientific discipline.” Programs may also find that that the most critical issues affecting the success of their students in specific areas are related to college-wide or GE outcomes. In this case, these outcomes may be chosen for the annual assessment reporting via OATF forms. The key part of the process is that program faculty and staff identifies the most important learning issues affecting the success of their students and focuses their assessment reporting efforts on outcomes related to those issues. Brief reporting forms will be collected each semester. Samples of the OATF 50 forms can be found in the appendix to this handbook. The process consists of two stages in each academic year: Fall Semester: At the beginning of the fall semester programs will be asked to identify the most important learning issues facing their students and the outcomes that align with those issues. More than one issue may be identified. Programs will focus their outcomes assessment reporting for the year on those outcomes. These may be college-wide, GE, program, course, or activity outcomes. Each program will complete a brief form that describes the following: • the important issues identified by the program faculty and staff • the outcomes associated with those issues • a summary of plans to assess, and report the assessment of, those outcomes Spring Semester: In the spring programs will report on their assessments of the outcomes identified in the fall. They will complete a brief form that reports the following • the important issues and related outcomes identified in the fall • the methods used to assess those outcomes • the results of the assessments • what was learned from that assessment; i.e. what was identified as effective and what can be changed to improve student success based on the fall assessment 51 Using the results of general college-wide and GE outcomes Data gathered to assess the college-wide and GE outcomes provide vital information about the cumulative effect of students’ experiences at the college. As such, it will be of interest to many groups across campus and all college faculty and staff. It will inform our campus planning efforts in many ways. A college-wide dialogue, including all members of the college community will help us understand the implications of what we learn from our assessment of college outcomes and how we can improve what we do to assist students in achieving those outcomes. The college Research Office and CASSL will work to compile and disseminate the results of the assessment of college-wide and GE outcomes and provide opportunities for dialogue. This could include: • Sending the results to each shared governance committee for dialogue and, potentially, recommended actions. • Reporting to the entire CRC faculty and staff via CASSL workshops, discussions, and publications and Research Office publications. • Including the information as part of the data packages that are provided to all programs undertaking program review. • Providing opportunities for faculty who teach GE courses to review and discuss the GE outcomes assessment. The results of college-wide and GE outcomes assessment, disseminated to the entire college community, could be used to improve processes across the college, both in and out of the classroom. Information from the assessment of college-wide and GE outcomes will inform many of our planning processes including • Department planning and curriculum review • Unit planning • College strategic planning • GE course review • Review of College-wide outcomes Many of the changes listed in earlier chapters of this handbook as resulting from the assessment of course/activity or program outcomes could also occur as the result of dialogue surrounding the assessment of college-wide and GE outcomes. 52 Appendix: Sample OATF Forms The cover page for the FALL 2008 draft forms is shown below followed by several examples of forms filled out for hypothetical programs. The Spring forms will be added to the appendix as they become available. 53 54 Fall SLO Dialogue – Agenda and Summary Cover Page In order to make our use of student learning outcomes valuable and constructive, let’s focus on assessing learning related to those issues identified as most important by each program. The following form asks you, as a program, to identify the key learning issues that are impacting your students. You can choose to focus on any learning issue that is affecting your students. Some examples: o Skills or abilities specific to your program: o Student understanding of core program concepts or processes o Skills that are essential to student success following program completion o Abilities that students struggle to master o Skills needed for gate-keeper processes or courses o Tasks essential to student progress through the steps of the program o General education outcomes important to success for students in your program o College-wide outcomes important to success for students in your program. In order to make this process more flexible for the many things we do at CRC, three types of programs are identified. [A list of programs is available from the college Research Office]. Instructional Programs: These are programs in which the major interaction between students and CRC faculty/staff occurs in credit-granting courses. Examples include programs from Business to Social Science and departments from Accounting to Work Experience. In the catalogue, these are usually called “Areas of Study”. Non-instructional Programs - Student Services/Student Support Programs: These are programs in which CRC faculty/staff work directly with students in a variety of settings other than credit granting classes. The catalogue defines student support services as those which “…assist students with career decision-making, problem resolution, and goal achievement” (catalog p.9). This includes a wide range of services such as Assessment Services, Counseling, DSPS, EOPS, CARE, Heath Services, Library, Math Center, Reading Writing Center, MESA/CCP, Student Development, Tutoring, etc. These programs may have Service Area Outcomes instead of, or in addition to, Student Learning Outcomes. Non-instructional Programs - Instructional Support/Administrative Services Programs: These are program that indirectly support teaching and learning. They include Admissions and Records, Bookstore, Business Services, CASSL, Research Office, Custodial Services, Instruction Office, etc. These programs may have Service Area Outcomes instead of, or in addition to, Student Learning Outcomes. Important notes: The faculty and staff of a program use their best professional judgment to identify learning issues important to the success of students in the program, to determine what outcomes reflect those critical issues, and to design assessments of those outcomes. o A subset of course SLOs can be assessed and reported each year; not all of the course SLOs need to be assessed and reported annually o One or a few program SLO(s) may be the focus of assessment one year; not all of the program SLOs need to be assessed annually. o SLO assessment does not require any specific research design; assessment may be based on narrative accounts, descriptive information and/or quantitative data and will often include multiple measures. o SLO assessment in instructional programs should be conducted for a sample of classes or students in a course, program or service area; however, SLO assessment does not mandate any minimum number of sections analyzed, does not require a particular method of assessment, and does not specify a minimum sample size. o An instructor is encouraged to save a record of the assessment privately to be able to describe the assessment to faculty in a department or program area. SLO assessment does not require the identification of individual instructors’ data in any reports or analyses. o Student Learning Outcomes (SLOs) complete the statement “students completing program activities will be able to…” . Service Area Outcomes (SAOs) complete the sentence “In order to support learning, CRC employees will…” 55 Fall SLO Dialogue – Agenda and Summary Instructional Programs Semester (e.g. Fall 2008) ____FALL 08____ Program: ____Oceanography________ 1. What are the most critical student learning issues impacting the success of students in your program? Such issues could include: student understanding of key program concepts, skills that are essential to success following program completion, abilities that students struggle to master, skills needed for subsequent courses in a pre-requisite sequence, etc. a. The ability to interpret data and draw reasonable conclusions from numerical or graphical information continues to be something that students struggle to achieve. b. Students in OCEAN 402 seem to be lacking one pre-requisite skill from OCEAN 401 (assessing the impact of air flow and ocean currents on near shore ecosystems). c. IMPAC has described a new list of content outcomes for transfer…student progress seems satisfactory on these, but we might want to look at it more closely in the future. 2. Which of these would you like to focus on at a program level this fall? What program and/or course Student Learning Outcomes are related to the selected issue(s)? A” is closely aligned with Program Outcomes #2… “B” is directly related to the course outcomes # 3 for OCEAN 401. 3. What might be done or is being done to facilitate student success in this area? (E.g. effective strategies currently being utilized in the program or that you plan to implement.) a. Increased use of in-class active learning techniques and problem-based homework assignments may help students learn to interpret data, including that from numerical or graphical data. b. Dialogue among full time and adjunct faculty teaching OCEAN 401, including shared examples of class exercises and homework problems, may help. A common set of problems that could be used to teach this skill could be developed. An exit assessment at the end of OCEAN 401 may be useful. 4. How might you assess the effectiveness of these strategies? Please mark all that apply □ Direct observation of student actions or performance (e.g. student skills during theatre performances or athletic events, structured practice or drills, student work in laboratory or studio classes, results of “practical” exams, etc.) X Assessment of student products (e.g. review of semester projects, grades on inquiry projects, etc.) □ Analysis of student creative work (e.g. art portfolios, creative projects in classes, student projects as part of a campus cultural event, etc.) X Exams, quizzes, and/or problem sets in courses (items linked to specific outcomes) X Grading of assignments based on rubrics (essays, reports, case studies, etc.) □ □ □ Student self-assessments (reflective journals, surveys, etc.) Capstone projects or final summative assessment (e.g. final exams, capstone projects, etc.) Guided discussion (group or individual) X Other (please describe) – Intake assessments of students in the first week of OCEAN 401 to assess the skills with which they enter the course. Course Summary Note: In the spring, you will be asked to check off those courses in which learning has been examined to evaluate and improve student success on course outcomes. 56 Fall SLO Dialogue – Agenda and Summary Instructional Programs Semester (e.g. Fall 2008) ___Fall 2008_ Program: ____Health Care for the Elderly__ 1) What are the most critical student learning issues impacting the success of students in your program? Such issues could include: student understanding of key program concepts, skills that are essential to success following program completion, abilities that students struggle to master, skills needed for subsequent courses in a pre-requisite sequence, etc. A revision of the certification exam for “Providers of Health Care for the Elderly” has resulted in a need to revise curriculum in some classes. Dialogue among program faculty has indicated that students struggle with outcomes related to conducting interviews to assess the health needs of frail elderly people. 2) Which of these would you like to focus on at a program level this fall? (You can choose more than one). What program or course Student Learning Outcomes are related to the selected issue(s)? o Curriculum revisions are in progress; assessment of outcomes for revised courses will occur next fall, so we will focus on improving student success on the outcomes related to interviewing the frail elderly in order to assess their health needs. These outcomes can be found in the course outlines for HCE 100: Current Issues in Health Care for the Elderly, and HCE 130: Assessment of Health Care Needs for the Frail Elderly. 3) What might be done to facilitate student success in this area? Include effective strategies currently being utilized in department in this discussion. o Increased use of problem-based learning assignments related to focal outcomes. o Revision of materials for students related to the focal outcomes. o Changes in labs during which interpersonal interview skills are practiced. 4) How might you assess the effectiveness of these strategies? Please mark all that apply X Direct observation of student actions or performance (e.g. student skills during theatre performances or athletic events, structured practice or drills, student work in laboratory or studio classes, results of “practical” exams, etc.) X Assessment of student products (e.g. review of semester projects, grades on inquiry projects, etc.) □ Analysis of student creative work (e.g. art portfolios, creative projects in classes, student projects as part of a campus cultural event, etc.) X Exams, quizzes, and/or problem sets in courses (items linked to specific outcomes) X Analysis of student work on problem sets, homework, etc. □ □ □ □ □ Grading of assignments based on rubrics (essays, reports, case studies, etc.) Student self-assessments (reflective journals, surveys, etc.) Capstone projects or final summative assessment (e.g. final exams, capstone projects, etc.) Guided discussion (group or individual) Other (please describe) Course Summary Note: In the spring, you will be asked to check off those courses in which learning has been examined to evaluate and improve student success on course outcomes. 57 Fall SLO Dialogue – Agenda and Summary Instructional Programs Semester (e.g. Fall 2008) _FALL 2007_____ Program: _____Pharmacology ___________ 1. What are the most critical student learning issues impacting the success of students in your program? Such issues could include: student understanding of key program concepts, skills that are essential to success following program completion, abilities that students struggle to master, skills needed for subsequent courses in a pre-requisite sequence, etc. Abilities that students struggle to master o proper use of significant figures o logical development to problem solving o naming chemical compounds o manipulating formulas to solve problems 2. Which of these would you like to focus on at a program level this fall? (You can choose more than one). What program or course Student Learning Outcomes are related to the selected issue(s)? Proper use of significant figures P-SLO 1 Understand the Scientific Method which includes the ability to: o Interpret, manipulate and quantify generated data 3. What might be done to facilitate student success in this area? Include effective strategies currently being utilized in department in this discussion. o Continual reinforcement/reminders of proper use of significant figures o every time a problem is worked on the board, the rules for significant figure use are covered o Frequent checks of understanding (oral quiz, call and response, hard copy quiz/exams, lab reports) followed by reinforcement of proper use o Group activities allowing students to teach each other proper use of significant figures 4. How might you assess the effectiveness of these strategies? Please mark all that apply √Direct observation of student actions or performance (e.g. student skills during theatre performances or athletic events, structured practice or drills, student work in laboratory or studio classes, results of “practical” exams, etc.) □ □ Assessment of student products (e.g. review of semester projects, grades on inquiry projects, etc.) Analysis of student creative work (e.g. art portfolios, creative projects in classes, student projects as part of a campus cultural event, etc.) √ Exams, quizzes, and/or problem sets in courses (items linked to specific outcomes) □ Grading of assignments based on rubrics (essays, reports, case studies, etc.) √ Student self-assessments (reflective journals, surveys, etc.) □ □ □ Capstone projects or final summative assessment (e.g. final exams, capstone projects, etc.) Guided discussion (group or individual) Other (please describe) Course Summary Note: In the spring, you will be asked to check off those courses in which learning has been examined to evaluate and improve student success on course outcomes. 58 Fall SLO Dialogue – Agenda and Summary Instructional Programs Semester (e.g. Fall 2008) ___Fall 2008____ Program: _Forensics - Speech and Debate 1. What are the most critical student learning issues impacting the success of students in your program? Such issues could include: student understanding of key program concepts, skills that are essential to success following program completion, abilities that students struggle to master, skills needed for subsequent courses in a pre-requisite sequence, etc. o Students are lacking the ability to construct logical reasoning in their arguments for debate cases, particularly in setting forth a criteria or key premises to uphold and support their case. o Students are lacking in the ability to document substantial evidence in their debate cases that has a breadth or depth of research and/or support for arguments that are constructed and presented during a debate. o Students new to the “Forensics” speech and debate team seem to need more time or exposure to coaching to receive more training for the details and complexity of debate case construction. 2. Which of these would you like to focus on at a program level this fall? (You can choose more than one). What program or course Student Learning Outcomes are related to the selected issue(s)? o Students should be able to design and relate messages clearly for effective and appropriate oral communication (Program SLO #1). o Students should be able to analyze and formulate critical thinking within the evidence and reasoning of spoken and written messages (Program SLO #4). 3. What might be done to facilitate student success in this area? Include effective strategies currently being utilized in department in this discussion. Currently students are required to meet with an instructor, forensics team coach or experienced team captain for a certain number of hours as part of the course assignments to earn credit for their courses. Per our discussion today, in our class sessions of Communication 373 and in our team meetings (workshop sessions), it will be useful to have more experienced debaters and presenters “pair off” with the newer or more novice debaters and team members for more practice debates and as they work on researching for evidence to build their debate cases. In classes, there needs to be more elaboration on how to develop a criteria (weighing mechanism or set of standards) when presenting and arguing a case in a debate. 4. How might you assess the effectiveness of these strategies? Please mark all that apply □ X Direct observation of student actions or performance (e.g. student skills during theatre performances or athletic events, structured practice or drills, student work in laboratory or studio classes, results of “practical” exams, etc.) □ □ Assessment of student products (e.g. review of semester projects, grades on inquiry projects, etc.) Analysis of student creative work (e.g. art portfolios, creative projects in classes, student projects as part of a campus cultural event, etc.) □ □ □ □ □ □ X Exams, quizzes, and/or problem sets in courses (items linked to specific outcomes) X Grading of assignments based on rubrics (essays, reports, case studies, etc.) Student self-assessments (reflective journals, surveys, etc.) Capstone projects or final summative assessment (e.g. final exams, capstone projects, etc.) X Guided discussion (group or individual) Other (please describe) 59 Fall SLO Dialogue - Agenda and Summary Student Support / Student Services Programs Semester (e.g. Fall 2008) _Fall 2008__ Program: __ LESA (Languages and English Studies Achievement) 1. What are the most critical student learning issues impacting the success of students in your program? You might want to consider student understanding of key program processes, tasks essential to student success, abilities that students struggle to master, skills needed to progress through the steps of the program, etc. o Many students who be eligible for and would benefit from LESA are not aware of the program. o Students have difficulty completing the forms for the national LESA scholarships. o The ability to effectively use the tutoring program at the LESA center is difficult for many students. 2. Which of these would you like to focus on at a program level this fall? What Student Learning Outcomes and/or Service Area Outcomes are related to the selected issue(s)? o This is closely aligned with program outcome #1 o This is closely aligned with activity outcomes #3 o This issue is under study long term; we will focus on the tutoring program next year. 3. What might be done or is being done to facilitate student success in this area? (E.g. effective strategies currently being utilized in the program or that you plan to implement.) o Outreach efforts can be increased by providing additional materials at orientation, visiting beginning English and Foreign Language classes, and updating the website. o All LESA faculty and staff and LESA student coordinators and tutors should practice filling out the scholarship forms. A step-by-step guide to filling out the form can be developed. 4. How might you assess the effectiveness of these strategies? Please mark all that apply Direct observation of actions or performance (e.g. interactions during a counseling visit, activities during a club meeting, etc.) X Assessment of student task results (e.g. completeness of student financial aid forms, student success at registration at orientation, etc.) Analysis of student creative work (e.g. student projects as part of a campus cultural event, etc.) Student self-assessments (reflective journals, surveys, etc.) Guided discussion (group or individual) X Other (please describe): interviews with students signing up for LESA to determine how they heard about the program. 60 Fall SLO Dialogue - Agenda and Summary Student Support / Student Services Programs Semester (e.g. Fall 2008) _Fall 2008__ Program: __Student Mental Health Clinic 1. What are the most critical student learning issues impacting the success of students in your program? You might want to consider student understanding of key program processes, tasks essential to student success, abilities that students struggle to master, skills needed to progress through the steps of the program, etc. o Students referred to the clinic often misunderstand the services that are available and/or are confused about their need for the services provided by the MHC. o Student use of the clinic rises sharply in the two weeks preceding finals, overstretching the ability of MHC staff to effectively see all students in a timely fashion. 2. Which of these would you like to focus on at a program level this fall? What Student Learning Outcomes and/or Service Area Outcomes are related to the selected issue(s)? o These two issues are both related to Program SLO 2 “Students will become effective at recognizing their need for mental health services and appropriately choosing to access the MHC”. o The second issue is related to SAO 1 “Staffing will be scheduled to meet student demand” 3. What might be done or is being done to facilitate student success in this area? (E.g. effective strategies currently being utilized in the program or that you plan to implement.) o Outreach efforts can be increased by providing additional materials at orientation, visits to classrooms to present information, additional training for all staff referring students to MHC. o Scheduling problems might be addressed by allowing flexible work hours throughout the semester, by hiring student assistants, and by seeing some student clients in groups when appropriate. 4. How might you assess the effectiveness of these strategies? Please mark all that apply Direct observation of actions or performance (e.g. interactions during a counseling visit, activities during a club meeting, etc.) Assessment of student task results (e.g. completeness of student financial aid forms, student success at registration at orientation, etc.) Analysis of student creative work (e.g. student projects as part of a campus cultural event, etc.) X Student self-assessments - surveys of student awareness of the MHC services X Guided discussion – focus groups of students to identify the common perceptions of mental health services among various student groups. X Other (please describe): Analysis of records of clinic demand and client-to-staff ratio at various times in the semester 61 Fall SLO Dialogue Agenda and Summary Instructional Support / Administrative Services Programs Semester (e.g. Fall 2008) Fall 2008 Student Leadership Program: TASSL - Team for the Advancement of Staff and 1. What are the most critical student learning issues affected by the staff of your program? You might want to consider student understanding of key program processes, tasks essential to student success, abilities that students struggle to master, skills needed to progress through the steps of the program, etc. o Professional development activities related to best leadership practices have for the most part been effective; however, additional TASSL activities related to cultural competence are needed. Evaluation forms completed by CRC staff and students involved in TASSL activities indicate that this area is rated lower than the others. 2. Which of these would you like to focus on at a program level this fall? What Service Area Outcomes are related to the chosen issue(s)? [Service Area Outcomes (SAOs) complete the sentence “In order to support learning, CRC employees will…”] o This issue is directly related to Program Outcome 1: As a result of TASSL activities CRC staff and students in leadership positions will be able to demonstrate cultural competence, work to reduce bias on campus, and engage in productive collegial interactions across the college 3. What might be done to facilitate student success in this area? Include effective strategies currently being utilized in department in this discussion. o Recruitment of CRC staff with expertise in anti-bias work, diversity training, and cultural communication to co-sponsor TASSL events. o Additional professional development for TASSL staff on best practices in supporting cultural competence at college campuses. 4. How might you assess the effectiveness of these strategies? Please mark all that apply Direct observation of program processes (e.g. record keeping processes, staff training processes, activities during program meetings, etc.) Assessment of student work indirectly related to the program (e.g. completeness of student financial aid forms, etc.) X Staff assessments of program effectiveness Analysis of program records X Student self-assessments of their interactions with program processes (surveys, etc.) Guided discussion among program employees (group or individual) X Other (please describe) Increase in the number of TASSL events that focus on cultural competence accompanied by the use of a modified evaluation form for TASSL events. 62 Fall SLO Dialogue Agenda and Summary Instructional Support / Administrative Services Programs Semester (e.g. Fall 2008) Fall 2008 Services Program: Campus Placement Agency–Temp Staffing 1. What are the most critical student learning issues affected by the staff of your program? You might want to consider student understanding of key program processes, tasks essential to student success, abilities that students struggle to master, skills needed to progress through the steps of the program, etc. A. Students have difficulty with completing the forms needed for job application across the district, reducing the number successfully placed in temporary jobs. B. Staff often assist students with resume development. Staff keep records of student temporary employment to assist with this task. Pulling records for students has become a very time consuming task for program staff. 2. Which of these would you like to focus on at a program level this fall? What Student Learning Outcomes and/or Service Area Outcomes are related to the chosen issue(s)? Note: Student Learning Outcomes complete the statement “students completing program activities will be able to” Service Area Outcomes (SAOs) complete the sentence “In order to support learning, CRC employees will…” o Both of these issues will be a focus of our work this year. Issue “A” is aligned with Program SLO 1, “students will successfully apply for and obtain temporary jobs”. Issue “B” is reflective of both SAO 2 “Staff will assist students with resume development” and SAO 3 “Staff will keep accurate records of student employment”. 3. What might be done or is being done to facilitate student success in this area? (E.g. effective strategies currently being utilized in the program or that you plan to implement.) o Development of workshops and/or handouts for students that teach them in how to fill out the forms. o Development of a process by which a student can access his/her own job records electronically. Note: This will require funding and/or time for a programmer to develop such a process. o Providing job record forms for students and training them to keep their own employment records. 4. How might you assess the effectiveness of these strategies? Please mark all that apply Direct observation of program processes (e.g. record keeping processes, staff training processes, activities during program meetings, etc.) X Assessment of student work indirectly related to the program - e.g. review of completed job application forms for accuracy. X Staff assessments of program effectiveness – assessment of time needed for record production. Analysis of program records X Student self-assessments of their interactions with program processes – surveys of students confidence in their ability to keep their own job records. Guided discussion among program employees (group or individual) Other (please describe) 63
© Copyright 2026 Paperzz