Annual Assessment Report to the College 2010-­‐11 College: Mike Curb College of Arts, Media, and Communication Department: Journalism Program: Bachelor of Arts Note: Please submit your report to (1) the director of academic assessment, (2) your department chair or program coordinator and (3) the Associate Dean of your College by September 30, 2011. You may submit a separate report for each program, which conducted assessment activities. Liaisons: Linda Bowen (Fall 2010) and Taehyun Kim (Spring 2011) 1. Overview of Annual Assessment Project(s) 1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the program this year. Journalism’s program assessment coincided with participation in the CSUN “Simplifying Assessment Across the University” pilot project. The project’s components, included: identifying courses that fit the “gateway” and “capstone” definitions, developing embedded signature assignments within those courses, designing rubrics to measure the student-­‐learning outcomes, and compiling and analyzing the results. The gateway selection was JOUR 210, News Writing and Reporting II, which is a required lower division core skills course for majors. o For the signature assignment, students completed a news story covering a civil or criminal court case. The common and embedded news reporting and writing assignment was administered to 122 students in six sections of JOUR 210 in Fall 2010. Both full-­‐ and part-­‐time faculty members were involved. They provided input and support to ensure that students’ work was completed appropriately and uploaded to the project’s electronic assessment system website. The capstone selection proved more difficult to determine. See 1b for explanation. In addition, the Department assessment committee had planned to: (1) revamp the Internship Program evaluation form used by workplace site supervisors, with the goal to more effectively assess student learning and the internships’ relationship to the Department’s Student Learning Objectives, and (2) analyze the required intern-­‐generated written reports and logs. They represent a so-­‐far untapped rich source of data about the workplace and students’ experiences. July 18, 2011, Bonnie Paller Is assessment under the oversight of one person or a committee? o Since Spring 2009, assessment has been the responsibility of a three-­‐person committee, including the Academic Assessment Liaisons Committee representative. However, major decisions and most assessment activities have involved the entire full-­‐time faculty, (a relatively small unit at 11), as well as a few adjunct faculty members who teach the skills courses selected for assessment. For example, several full-­‐time and adjunct faculty members were involved in designing and assigning the “signature assignment” in JOUR 210 and supervising the uploading of their students’ documents to the pilot Electronic Assessment System website repository, which was part of the university-­‐wide “Simplifying Assessment” project mentioned above. For more than 10 years, Journalism has had a standing assessment committee with a permanent place on the monthly faculty meeting agenda, providing the committee chair/liaison with a platform for discussions, projects and other relevant activities. 1b. Implementation and Modifications: Did the actual assessment process deviate from what was intended? If so, please describe any modification to your assessment process and why it occurred. The actual assessment process deviated from the plan in several ways and for a variety of reasons, including progress made on the Journalism Department’s strategic planning process, which has been driven by the ongoing assessment activities. o Simplifying Assessment Across the University began as a multi-­‐faceted pilot, with an electronic website component created by project participants from the University’s Institutional Research. As with any pilot, it takes time to develop content, test ideas and concepts and adjust as needed. For Journalism, this project did not fit neatly into the pilot timeline. The original plan was to create a news exam that would be administered to students in both the gateway and capstone courses. Faculty members had for some time discussed the feasibility of using an industry-­‐related practical “qualifying” exam to establish a baseline from which to measure student learning in standardized journalistic elements, such as news judgment, style/copy editing and writing, grammar, punctuation and basic news story construction. In this scenario, students would take the exam through Moodle. A normed rubric would be applied to their answers, coinciding with pre-­‐determined goals and benchmarks. Then, at the capstone level, the news exam would be re-­‐administered to determine and measure student learning since the gateway exam. Flaws in that concept, coupled with the pilot’s focus on assessment of writing, altered the plans. Instead, the focus shifted to using the news reporting and writing assignment in JOUR 210, as noted above. The next step will be to finalize the scoring rubric, get it into the electronic system, conduct the actual evaluation of students’ work online and analyze the results during Fall 2011. o Identifying an appropriate “capstone” course that could produce meaningful results proved difficult and delayed assessment, originally intended for Spring 2011. Faculty initially chose JOUR 400, Mass Communication Law and Ethics, a required core upper division course, but later rejected it when they realized too many students take JOUR 210 and JOUR 400 simultaneously. JOUR 498, the senior tutorial, which all journalism majors are required to take in their graduating semester, was identified as a more appropriate capstone course despite the varying topics offered in the tutorials. But, while the “news exam” might be a suitable vehicle to establish a baseline of our students’ knowledge, it might not adequately capture what they have learned by the time they are ready to graduate. Faculty, during the strategic planning process, decided on a more representative “culminating experience.” In this model, the Department’s recently July 18, 2011, Bonnie Paller revised learning outcomes would be applied to a representative portfolio of student-­‐produced work in the various emphases – print, broadcast, photojournalism and public relations – and measured, using an applicable set of rubrics containing common news media storytelling objectives and outcomes. Now, the capstone assessment is scheduled for Spring 2012 as part of the Department’s new Program Assessment Plan, 2011-­‐2016. 2. Student Learning Outcome Assessment Project: Answer questions according to the individual SLOs assessed this year. If you assessed more than one SLO, please duplicate this chart for each one as needed. 2a. Which Student Learning Outcome was assessed this year? For the Simplifying Assessment pilot, the focus was on SLOs 1 and 2, because of the shift from a more global news exam to a common “signature” news reporting and writing assignment in a single course (see 1b.). Note: all six of the Journalism Department SLOs have been revised as a result of the ongoing assessment and curriculum revision activities in the strategic planning process. The revised SLOs are not reflected here. 1) Attain competency in writing basics such as grammar and punctuation, word usage and spelling, sentence and story structure and journalistic style; 2) Attain competency in the gathering and critical analysis of information using such techniques as interviewing, observation and researching primary and secondary sources. 2b. What assessment instrument(s) were used to gather evidence about this SLO? As noted previously in this report, the common “signature” news reporting and writing signature assignment of covering a civil or criminal court case was used as the main instrument, along with the companion “Rubric for News Story Assignment.” 2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of students, which courses, how decisions were made to include certain participants. In Fall 2010, 77 students uploaded their court beat stories (assignments) to the Simplifying Assessment Electronic Assessment System. The sample represented nearly 65 percent of the total number (122) of JOUR 210 students enrolled in six sections of the course. JOUR 210 repeatedly has been used as a focus of assessment because all of our students have to take the course. The bulk of them do so on our campus, since few Community Colleges offer the equivalent for transfer. The majority of students in the multiple sections of this course are juniors, but a few sophomores and seniors may also be enrolled in any given semester. Also, the public affairs-­‐related curriculum requires certain common assignments, such as covering city hall, police and fire, or courts, in addition to other styles and types of journalism. July 18, 2011, Bonnie Paller 2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-­‐sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used. The “Simplifying Assessment Across the University” project will eventually provide the mechanisms for a variety of kinds of design methodology, including longitudinal and cross-­‐sectional of student work in the lower division core skills course. However, in this pilot year, the use of a “signature” embedded news reporting and writing assignment was designed simply to establish a baseline from which to start to build. Similar assignments in this same JOUR 210 course have been used in the past as the assessment focus, which has mainly been on style, grammar and punctuation and news-­‐gathering rather than other standardized journalistic elements, such as news judgment or storytelling. 2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the collected evidence. The results and analysis for the Simplifying Assessment project remain unfinished, primarily because the gateway portion of the pilot is expected to be completed in Fall 2011, followed by the capstone in Spring 2012 as noted in the new Program Assessment Plan, 2011-­‐2016. However, the project’s progress and revisions generated meaningful discussions among the faculty about the need for gateway/capstone courses or experiences that can be measured. The concepts behind the “Simplifying Assessment” project have been part of the Strategic Planning Committee discussions as well. 2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program, student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc. Please provide a clear and detailed description of how the assessment results were or will be used. The Journalism faculty has been engaged in an examination of its entire program, manifested in its strategic planning and curriculum revision processes, which were the direct result of a variety of formal and informal, direct and indirect assessment results. These activities have produced tangible changes even as these groups continue their work into 2011-­‐12. For example, program SLOs have been revamped with more explicit language on skills acquisition applications and demonstrations as well as inclusion of knowledge related to transformations in the profession. Additional results include: o Collaboration with the Department’s curriculum committee on appropriate curricular changes, i.e., adding new experimental multimedia courses, such as Backpack Journalism and Muslims in the Media (both new in Fall 2011), New Media and Spanish-­‐Language Television, and incorporating convergence-­‐based tools and techniques instruction into existing skills courses; o Adoption of curricular changes in the introductory skills course, JOUR 110, to include beginning multimedia storytelling tools and techniques. A series of meetings held among the full-­‐ and part-­‐time faculty who teach the key core skills courses resulted in consensus on the level of what should or should not be taught in these existing courses, and pointed to a variety of related topics to address in the July 18, 2011, Bonnie Paller o
o
continuing revamping of the curriculum, integrating relevant tools and technologies as appropriate across the courses; Development of a draft strategic, long-­‐range planning document, embracing three visions – for students, faculty and the Department. The planning committee members recognized that a vital element of this process is connected to the relationship between curriculum development and assessment results/activities. Thus, the committee used the Student Learning Objectives as the foundation for its planning process in the student vision area, aligning the strategic plan to the program SLOs, with proposals to institute the aforementioned capstone/culminating senior project-­‐like experience, and focusing on student media as the hub of teaching and learning; Provided all faculty members with resources needed to ensure that their courses have SLOs and that all faculty members include them, along with the departmental SLOs, in their syllabi. The packet, containing information about existing course SLOs and instructions on how to write effective SLOs, using active verbs and other characteristics, was distributed to faculty before the start of the Fall 2011 semester. 3. How do this year’s assessment activities connect with your program’s strategic plan and/or 5-­‐yr assessment plan? This year’s assessment activities has been the foundation of the Journalism Department’s Strategic Planning process, particularly as the “Simplifying Assessment” pilot has emerged, with its focus on establishing gateway/capstone courses as well as the “signature” assignments within these courses. In addition, the move, beginning in Spring 2011, for a more cohesive college assessment process has enhanced the link between the strategic plan and the Program Assessment Plan, 2011-­‐2016. Journalism, like its CAMC counterparts, linked its five-­‐year Program Assessment Plan to a CAMC-­‐based adaptation of the “Simplifying Assessment” model in a project spearheaded by the college assessment director and associate dean in early Spring 2011. That process of drawing the six liaisons together preceded creation of a new set of six College Learning Objectives, presented to CAMC department chairs in Fall 2011 – the next step in the maturation of assessment within the college. It coincides with the creation of CAMC 100, and is expected, as the bonds evolve, to foster sustainable assessment practices that link department and college objectives. Also, it should provide a forum for interdisciplinary collaboration and communication. 4. Overall, if this year’s program assessment evidence indicates that new resources are needed in order to improve and support student learning, please discuss here. The shift to include college-­‐based assessment might create a need for workshop/training funding for all faculty members, particularly in developing assessment tools for “capstone”-­‐type experiences and courses. 5. Other information, assessment or reflective activities not captured above. July 18, 2011, Bonnie Paller Two other activities were part of the Department’s assessment considerations: o The Program Assessment Plan for the next five years was distributed to the CAMC associate dean and to the University Assessment Office in May 2011. That plan is part of a unified college assessment process undertaken in Spring 2011. It incorporates an adaptation of the university “Simplifying Assessment” pilot model, which was introduced to the six department college liaisons and CAMC Academic Council members in a planning workshop held March 22, 2011, and adopted as the template used by each of the six assessment liaisons for their departments’ five-­‐year plans. o Overhaul of the Journalism Internship Program to coordinate with university-­‐wide curriculum classification changes, as well the assessment-­‐driven need to evaluate overall curriculum and instruction beyond the typical and traditional analysis of data derived from internship evaluation forms completed by site supervisors. Management of the internship program was centralized, with one adviser to better supervise and coordinate the experience for the 10 percent of Journalism students who take the internship course. In addition, the internship supervisor, along with a faculty colleague, organized workshops, migrated the biweekly reports to Moodle, and assigned additional reading to support the internship experience. Now, students are required to attend three workshops throughout the semester, upload all assignments through Moodle, and read a textbook about maximizing the internship experience. More changes are expected with the university-­‐wide move to a web-­‐based internship evaluation tool. 6. Has someone in your program completed, submitted or published a manuscript, which uses or describes assessment activities in your program? Please provide citation or discuss. N/A July 18, 2011, Bonnie Paller