AI-generated triptych: on the left, inside a series of circles of bright lights in motion sits a disembodied head that is primarily made of wires but has a human female face; on the right a picture of an unreadable flier with black text on white background in text boxes, a pale blue oval in the center, and dark text on a dark beige background. On the right is another flyer, similar to those used to advertise an event--it features a cartoonish drawing of a woman's head dominating the center of the page; a bottom footer section features several circles with blurry images of heads. The flyer is dark grey with a peach colored center running diagonally through it; the (unreadable) text is black. A white-colored title proclaims 'Gendvatve Gngrative Ail Tertergolrlogies.'

Reconsidering Writing Pedagogy in the Era of ChatGPT: Results of a Usability Study of ChatGPT in Academic Writing

Lee-Ann Kastman Breuch University of Minnesota
Kathleen Bolander University of Minnesota-Twin Cities
Alison Obright University of Minnesota-Twin Cities
Asmita Ghimire University of Minnesota-Twin Cities
Stuart Deets University of Minnesota-Twin Cities
Jessica Remcheck University of Minnesota-Twin Cities

Introduction

The emergence of Generative Artificial Intelligence (AI) has produced both havoc and hope. Scholars in the field of composition have suggested that ChatGPT—a Generative AI tool—has changed the dynamics of teaching, writing and teaching writing (Baidoo-Anu & Owusu, 2023; Floridi, 2023). A recent statement by a combined MLA-CCCC task force suggested that ChatGPT would affect the creative and critical thinking of students, add more labor to teachers and instructors without compensation, and challenge the institutional policy and practices by asking it to redefine the idea of plagiarism. Composition scholars are also raising questions about the changing dynamics of writing, literacy, and pedagogical practices. For example, Johnson (2023) suggested that we redirect the threshold concepts of writing to tackle the present changing dynamics of teaching writing. Anson and Straume (2022) similarly encouraged instructors to work with AI tools rather than “combat or extinguish” them (p. 7).

In this chapter, we aim to extend scholarly discussions of generative AI tools by sharing results of a usability study of ChatGPT involving undergraduate students at a large midwestern university. We set out to learn how students were understanding ChatGPT texts and how they might consider using ChatGPT texts for academic purposes. Thirty-two student participants completed five academic writing tasks using ChatGPT, rated the ChatGPT texts, and answered debriefing questions. As our findings show, students had a rhetorical and multidimensional response to ChatGPT. They acknowledged its impressive capabilities but also raised key questions about using ChatGPT in several areas. Ratings showed that students were mostly satisfied with ChatGPT overall, but were less confident about using ChatGPT texts unaltered as their academic homework. Students raised questions about the credibility of ChatGPT texts, use of sources, integration of citations, tone and voice of ChatGPT texts, and how use of the text would impact their sense of scholarly identity and quality of learning. In sum, students expressed rhetorical critiques of ChatGPT while also sharing their impressions, both positive and negative.

Based on our qualitative and quantitative findings, we propose pedagogical principles that would help instructors and students engage with generative AI in writing and writing classes. The principles include expanding writing pedagogy to integrate generative AI in writing processes and developing critical AI literacy. We also provide screenshots and videos of ChatGPT sessions and highlight student comments to share perspectives of what students valued and questioned about ChatGPT.