OVERVIEW
In the context of writing, JMU faculty realized that students lacked an important writing-related skill: rhetorical awareness. With champions internal to the writing faculty, assistance from JMU’s Center for Assessment and Research Studies, and by strategic support from General Education Program administrators and senior administration, instructors made evidence-informed changes to their classes, resulting in demonstrable learning improvement. What follows not only serves as James Madison’s assessment report to the State Council of Higher Education for Virginia (SCHEV), but also tells the story of an institution-wide improvement effort.
Student Learning Outcomes (SLOs): What do we expect students to learn or do?
- Demonstrate an awareness of rhetorical knowledge, which may include the ability to analyze and act on understandings of audiences, purposes, and contexts in creating and comprehending text. (Student Learning Outcome of Focus)
- Employ critical thinking, which includes the ability through reading, research, and writing, to analyze a situation or text and make thoughtful decisions based on that analysis.
- Employ writing processes.
- Demonstrate an awareness of conventions, the formal and informal guidelines that define what is considered to be correct and appropriate in a variety of texts.
- Compose in multiple environments using traditional and digital communication tools.
What experiences lead to student learning?
All first-year students – approximately 4,700 admitted annually – are required to fulfill General Education program requirements related to writing. Approximately 70% of students do so by taking WRTC 103: Rhetorical Reading and Writing, JMU’s first-year writing course; the others either test out through AP scores or transfer credit in writing (e.g., dual enrollment). Recently, rhetorical awareness skills have become a focal point for the course, which is most closely related to SLO 1. Relatedly, the assessment processes that follow are tightly connected to rhetorical awareness.
How do faculty and staff know whether students have learned?
We began implementing a direct measure in 2019, which consists of two steps. The students are first assigned to an article. After reading the article, they complete a table to describe various rhetorical elements in the article and provide indicators from the text to support their descriptions. Five rhetorical elements are included in the measure:
- Purposes (what seems to have motivated the author to write this text? What does the author seem to want to accomplish? What does the writer seem to want readers to do?)
- Genre (what category of writing does this text represent? What are its features, such as style, format, and structure?)
- Author and publication (what does the article reveal about the author? What does the article reveal about the publication/publisher?)
- Audience (who seems to be the intended audience for the text?)
- Context (where and when was the article written? What is the historical, political, and/or cultural background for the article? Is there anything significant about the timing of the article?)
The measure then results in 10 scores, one for each combination of rhetorical elements (i.e., purpose, genre, audience, author/publication, and context) and process (e.g., describing the element and providing clues and indicators from the text to support the description). The four possible values for each score are 0 (left blank), 1 (“Beginning”), 2 (“Good”), and 3 (“Excellent,” indicating clear, accurate, and detailed responses). The total measure scores, therefore, range from 0 to 30, and can also be expressed as average scores across all 10 items, ranging from 0 to 3.
Were the institution’s expectations in these areas met?
Initially, NO. According to data collected among students enrolled in WRTC 103 in Spring 2021, students were not performing well with respect to rhetorical awareness, averaging a 1.6 out of 3 on the measure both before and after they completed the course. A 1.6 corresponds to a performance approximately mid-way between “Beginning” and “Good.”
How did the institution use this information to improve students’ educational experience and enhance future achievement?
In response to the aforementioned results, JMU implemented a university-wide initiative that included the following interventions:
Faculty development - The majority of the 40+ WRTC instructors:
- Reviewed the definition of rhetorical awareness, its components (e.g., genre), and the associated measure.
- Evaluated student performance on the measure.
- Explored different approaches of teaching rhetorical awareness from colleagues who had already piloted various approaches in their own classes.
- Incorporated rhetorical awareness into their individual classes (e.g., through lesson plans, activities, assignments, readings)
Changes to WRTC 103 classes - Students taking WRTC 103 subsequent to faculty development experienced the following changes:
- Repeated exposure to the measure (i.e., assessment as learning).
- Multiple weeks of exposure to, and practice of, rhetorical awareness skills.
Have changes made on the basis of previous assessment findings had the desired effect?
YES. After faculty were better prepared, student performance improved substantially. At the beginning of the Fall 2021 semester, students performed similarly to the Spring 2021 group (an average of 1.7). However, at the end of the semester – after receiving the revised curriculum – the same students’ average performance was a 2.0. In other words, our local evidence suggests the initiative paid off, helping students perform at the “Good” level on average. See graph below for the comparison of student performance before and after the learning improvement initiative.
What's Next?
JMU continues to emphasize writing as an essential skill for all students. The recent initiative related to rhetorical awareness is a noteworthy success. We are also aware that sustaining this initiative may be a challenge due to leadership and faculty turnover. JMU is working to solidify the rhetorical awareness initiative into the culture of WRTC 103, the General Education Program, and JMU’s assessment center.
More Information?
Dr. Caroline Prendergast’s Published Dissertation (coming soon). Contact fulchekh@jmu.edu regarding questions.