Annual Assessment Report to the College 2010-2011
College: Humanities
Department: English
Option: Literature
Committee Chair: Charles Hatfield
Note: Please submit report to your department chair or program coordinator and to the
Associate Dean of your College. You may submit a separate report for each program which
conducted assessment activities.
Liaison: Martin Pousson
1. Overview of Annual Assessment Project(s)
1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the
program this year. Is assessment under the oversight of one person or a committee? This year
the Literature Option’s assessment addresses the English Department’s common SLO #1,
students will demonstrate critical reading skills, using a five-point rubric, from 5 (excellent) to 1
(not demonstrated). The corpus for assessment consists of essays drawn from courses taught in
the most recent semester completed, Fall 2010: fifteen student essays drawn from multiple
sections of the Option’s gateway course, English 355 (Writing about Literature), and six essays
drawn from one section of the Option’s capstone course, English 495 (Senior Seminar in
English). This assessment process has been designed and solely implemented by the
Department’s Literature Committee, as is typically the case. Given the scope of the Option, the
Department may in future wish to consider more broadly distributing the work of Literature
assessment.
1b. Implementation and Modifications: Did the actual assessment process deviate from what
was intended? If so, please describe any modification to your assessment process and why it
occurred. The Literature Committee has carried out its assessment as outlined above. At an
early stage of the process, the Committee planned to assess an additional common SLO, #3,
students will demonstrate knowledge of creative, literary, linguistic, and/or rhetorical theories;
however, due to the difficulty of gathering an appropriate corpus (one relevant to the SLO and
also extensive and cross-sectional enough to yield meaningful results), the Committee decided
to defer this SLO to a future assessment. Note that for AY 2010-11 the Dept. encouraged all
Options to assess two SLOs each, one common and one Option-specific; however, Literature
does not have its own exclusive, Option-specific set of SLOs, since the Literature SLOs serve as
the foundational SLOs for all other Options. This is why the Committee considered assessing an
additional common SLO early on, though that proved too difficult to implement. Note that at
early stages the Committee discussed possible other means of assessment, but chose not to
adopt them; also, other student papers in addition to the corpus described above were
collected but not used (they have been retained for possible future use). The assessment
procedure, once agreed upon, did go smoothly, but note that the Committee spent time
considering, and continues to consider, multiple ways of meeting the University’s revised
assessment mandate.
2. Student Learning Outcome Assessment Project: Answer questions according to the individual
SLO assessed this year. If you assessed an additional SLO, report in the next chart below. N/A
2a. Which Student Learning Outcome was measured this year?
English Department Common SLO #1: Students will demonstrate critical reading skills.
2b. What assessment instrument(s) were used to measure this SLO?
A five-point rubric: 5 (excellent), 4 (more than satisfactory), 3 (satisfactory), 2 (less than
satisfactory), 1 (not demonstrated). This scale was crafted and agreed to by the whole
Department.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and
population size for this SLO. For example, what type of students, which courses, how decisions
were made to include certain participants. Student essays from the most recent semester
completed (Fall 2010) were rated per the above rubric. Fifteen essays were drawn from multiple
sections of the Option’s gateway course, ENGL 355 (Writing about Literature), and six essays
from one section of the Option’s capstone course, 495 (Senior Seminar in English). 355 and 495
were used for the sake of covering the Option cross-sectionally. Additional papers were
collected, including lengthy term projects from a 495 and past essays and exams from 313
(Studies in Popular Culture), but these were put aside in favor of using a more recent, relevant,
and uniform sample.
2d. Describe the assessment design methodology: For example, was this SLO assessed
longitudinally (same students at different points) or was a cross-sectional comparison used
(comparing freshmen with seniors)? If so, describe the assessment points used. The assessment
took the form of a cross-sectional comparison between ENGL 355, the gateway to the upperdivision English major, and 495, the capstone senior seminar for the English major. These are
the most readily identifiable assessment points in the major (intermediate assessment points in
the major are harder to identify, not because there are none but because there are so many
possible ones). Three members of the Literature Committee each rated the entire assessment
corpus, and then the results were compared and analyzed.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the data were
analyzed and highlight important findings from the data collected. The assessment yielded
three sets of results, here referred to as A, B, and C (one from each Committee member). Each
set includes data for the fifteen ENGL 355 papers and for the six 495 papers. The statistical
analysis of these sets is as follows:
ENGL 355—
A: Five papers were rated as less than satisfactory (2), four as satisfactory (3), four as more than
satisfactory, and one as excellent (5). The average score was 3.07 (rounded to two decimals);
the median score was 3; the mode was 2. (One paper was missing from this set.)
B: Five papers were rated satisfactory (3), six more than satisfactory (4), and four excellent (5).
Average: 3.93; median: 4; mode: 4.
C: Seven papers were rated satisfactory (3), five more than satisfactory (4), and two excellent
(5). Average: 3.53; median: 3; mode: 3.
All three: Average: 3.55; median: 3.5; mode: 3.
ENGL 495—
A: Four papers were rated less than satisfactory (2), one satisfactory (3), and one more than
satisfactory (4). Average: 2.5; median: 2; mode: 2.
B: Three papers were rated more than satisfactory (4), and three excellent (5). Average: 4.5;
median: 4.5; modes: 4 and 5.
C: Four papers were rated satisfactory (3), one more than satisfactory (4), and one excellent (5).
Average: 3.5; median: 3; mode: 3.
All three: Average: 3.5; median: 3.5; modes: 3 and 4.
Analysis of the overall averages from all three sets (A, B, and C) appears to indicate a satisfactory
to more than satisfactory level of achievement with regard to SLO #1 in both courses. However,
differences among the sets suggest that there may not be a firm consensus among the assessors
about how to evaluate critical reading. This is most obvious in the results for 495, where the
disparities are most pronounced.
Comparison of the 355 and 495 samples is complicated by the size differences between the
samples and the varied nature of the assignments that produced each sample; as a result, crosssectional analysis between the two may not be meaningful. In future the Literature Committee
may wish to consider initial norming (both within courses and cross-sectionally) in order to
establish consensus.
2f. Use of Assessment Results of this SLO: Think about all the different ways the results were or
will be used. For example, to recommend changes to course content/topics covered, course
sequence, addition/deletion of courses in program, student support services, revisions to
program SLO’s, assessment instruments, academic programmatic changes, assessment plan
changes, etc. Please provide a clear and detailed description of each. The average results from
all three sets suggest a satisfactory level of achievement for SLO #1. However, as noted, strong
differences among the sets suggest that there is not a solid consensus regarding this SLO and
that norming sessions and discussion may be needed in future to address this issue.
For comparison’s sake, the committees for the Subject Matter and FYI/JYI Options kindly
donated to the Literature Committee the results of their assessment in 355, which were based
on multiple batches of student essays drawn from two different points in the current term,
Spring 2011. This assessment was embedded in instruction and completed by three different
355 instructors (each instructor assessed her or his own students, rather than assessing a
common corpus). Despite notable differences in individual scoring, the Subject Matter and
FYI/JYI 355 results average about the same as the Literature Committee’s 355 results (≈ 3.65).
In any case, the Literature Option assessment described above is limited in that the two samples
assessed, 355 and 495, were incommensurate in size and represent different kinds of
assignments. In future the Option may wish to consider the possibility of embedded assessment
using common or similar assignments that make explicit the SLO(s) in question. Also, this
assessment, because it involves only small samples from two assessment points, is limited in
what it can reveal about the overall program. The Option may wish to consider more extensive
cross-sectional assessment that includes an intermediate point between gateway and capstone;
this might reveal needs for changes in requirements or course sequence. (At present the Option
has not identified any needs in those areas.) One complication here is that the undergraduate
English major has few means, other than personal advisement, of insuring course sequence: the
major has few prerequisites and no corequisites, and students may take courses out of the
expected sequence. Therefore it is difficult to identify with certainty an intermediate stage of
work, or even to insure that the gateway course, 355, serves as a gateway experience.
Some programs assess multiple SLOs each year. If your program assessed an additional SLO,
report the process for that individual SLO below. If you need additional SLO charts, please cut &
paste the empty chart as many times as needed. If you did NOT assess another SLO, skip this
section. N/A
3. How do your assessment activities connect with your program’s strategic plan?
The Literature Committee contributed to the Dept.’s strategic plan this AY by participating in the
revision of the common undergraduate SLOs so that they are based on measurable,
performance-based verbs (to facilitate the gathering of valid evidence) and by adopting the
Dept.’s five-point assessment rubric as described above. In future the Literature Committee will
of course participate in the Dept.’s effort, as stated in its new self-study, to address problems
discovered through the assessment process and to enhance the creative and scholarly
environment for students and faculty alike. Specifically, the Literature Committee will continue
to take part in the designing or revision of rubrics, the identification of assessment portals
(gateway, intermediate, and capstone), the brainstorming of improved assessment procedures,
and, per the Dept.’s strategic plan, the recruitment of students to majors and minors.
4. Overall, if this year’s program assessment evidence indicates that new resources are
needed in order to improve and support student learning, please discuss here.
Regarding SLO #1, the assessment results suggest that, though overall achievement levels
appear satisfactory (averages of 3.5-3.55), the Dept. should facilitate discussion regarding best
practices for teaching critical reading (obviously this would be relevant to all Options, not just
Literature). Further discussion of this issue among faculty could be very beneficial. In particular,
the Dept. has often debated the role and scope of its Literature gateway, ENGL 355, and an ad
hoc committee of 355 teachers in the next AY could do double or even triple duty by bringing
greater focus to the course, foregrounding critical reading (with potential benefits to all
Options), facilitating or embedding assessment in the course, and norming.
More broadly, the assessment experience this AY indicates that the Literature Committee needs
to solicit the input and participation of Literature faculty in assessment on a much broader scale,
a topic discussed in the attached appendix.
Finally, the Committee offers these observations: recent assessment experience indicates that
the Dept. must take a long-term approach to assessment, that assessment should be designed
and embedded in teaching from the start of the AY, that the work of the Dept.’s Assessment
Liaison is crucial, and that a two-year term for the Liaison would be optimal, so as to give the
Dept. continuity over an extended period. Despite pending cutbacks in the amount of
reassigned time available to the Dept., the Assessment Liaison position clearly warrants such
time.
5. Other information, assessment or reflective activities not captured above.
The Literature Committee faces serious challenges in designing and implementing assessment
procedures that truly reflect the Literature Option as a whole. Please see the attached appendix
(below, after Five-Year Plan)) for reflection on these challenges and on changes that the Dept.
may need to make going forward.
6. Has someone in your program completed, submitted or published a manuscript which
uses or describes assessment activities in your program? Please provide citation or
discuss.
N/A