Composing with AI

Pedagogical Implications

In this opportune moment, we would like to take a step back to reflect on how this exploratory usability study might inform the teaching of writing. Incidentally, we are all individually writing teachers, and we are keenly interested in the pedagogical implications of using generative AI. Indeed, we agree with the many writing scholars who suggested that ChatGPT provides an opportunity to “re-see” our writing pedagogy. In that spirit, and as writing teachers ourselves, we note some initial pedagogical implications for ChatGPT in writing classrooms. These implications include expanding writing pedagogy to integrate generative AI in writing processes and developing critical AI literacy. We also advocate a critical pedagogical perspective for implementing Generative AI technologies in writing classrooms.

Connecting Generative AI and Writing Processes

As we noted, findings of this study suggested that students appreciated the capabilities of ChatGPT for process-related activities including generating ideas, brainstorming, and outlining. Yet, students also articulated a number of questions about ChatGPT texts, such as doubts about information credibility, lack of depth, and ethical concerns about using ChatGPT. In response, we advocate creating opportunities for students to explore ChatGPT for process activities, but we also suggest centering a critical perspective in doing so. What might this look like? Pedagogical activities related to the writing process could involve facilitating all-class activities or exercises in which students are invited to use ChatGPT at various stages of their processes. The whole class approach is helpful for ensuring that all students have the opportunity to interact with and critically think about ChatGPT. Students could be asked to work in groups to explore ChatGPT, document its suggestions, and reflect on what is most useful and what is not useful about its contributions. Potential prompts to give ChatGPT might include brainstorming ideas on a specific topic or generating outlines. In these exercises, reflection is a key part, as it provides the opportunity for students to critically analyze ChatGPT as they have experienced it. Student groups might point out weaknesses and strengths of ChatGPT suggestions in all-class discussion. Table 1 includes specific suggestions for various process-based class activities that carry this critical perspective.

In addition, and as Graham suggested, ChatGPT strangely provides an opportunity for us to expand our understanding of process-based writing through a “post-process” lens that recognizes the unique attributes and functions of ChatGPT. According to Graham, a post-process pedagogy involving generative AI would view writing as an interaction with an LLM's corpus indirectly, and could center on the practices of fact-checking, curating, prompting, and revising. These practices could be built into classroom activities and assignments, such as by encouraging students to “fact-check” responses produced by ChatGPT, or examining the ways different prompts generate different responses, all with a critical eye. Ultimately, however, ChatGPT teaches us that critical AI pedagogy is to be centered—and not merely affirmed—in writing processes.

Practicing Critical AI Literacy

We find a critical perspective both helpful and necessary in thinking about the pedagogical implications of using ChatGPT. Critical pedagogy centers on the notion of critical consciousness, and here we consider what a “critical AI literacy” might look like. Such a stance would be open to an explicit awareness of generative AI technologies, but would also embrace a critical stance and perspective.

Two concepts come to mind that may help establish a “critical AI literacy.” The first concept is to consider technologies and ethics simultaneously. In Augmentation Technologies and Artificial Intelligence in Technical Communication: Designing Ethical Futures, Ann Hill Duin and Isabel Pedersen (2023) present an “ethical futures framework” that would “reframe professional practice and pedagogy to promote literacies surrounding the ethical design, adoption, and adaptation of augmentation technologies” (“Stronger Relationships, Stronger Programs: Asserting expertise for a generative AI Landscape”, 2023). This framework involves (1) knowledge of augmentation technologies (2) plans for building literacy capacity and (3) strategies for integrating ethical perspectives of adoption and practice. A second example concept involves a “post-digital critical pedagogy,” a pedagogical concept aiming to address the gap between designers and users. In “Postdigital Critical Pedagogy,” Petar Jandric and Sarah Haye (2021) described how this gap can exist: “...tech people did not care about critical pedagogy, and critical pedagogy did not care about tech” (p. 324). They advocated for designers and users to work together to better understand the internal workings of technology and to ask critical questions of it.

Both of these examples help us imagine “critical AI literacy.” Adopting an ethical framework in relation to ChatGPT might mean working to understand the technology and its capabilities while openly acknowledging the ways its use requires scrutiny and verification. In our study, students asked a number of critical questions about ChatGPT, such as “How does it pull information from the internet?”, “How much more advanced will this be in a couple of years?”, “Does it generate a different response for everyone?”, “What kind of database is it pulling from?” These questions showed that student participants are curious about the authenticity, credibility, transparency, and trustworthiness of AI. We see such questions as productive starting points for practicing “critical AI literacy,” and we encourage writing instructors to create spaces and activities in which students can ask these questions.

One exercise might be to ask students to apply an ethical framework to ChatGPT by discussing how and in what situations would various ChatGPT texts would be acceptable. Another exercise might include fact-checking and citation checking exercises which require students to use ChatGPT but also identify and check assertions and citations used in ChatGPT texts. This activity encourages critical reflection about ChatGPT assertions and citations that are seemingly confident; in checking assertions, students can directly question the accuracy of information included in ChatGPT texts. Another suggested exercise is to create requirements or rubrics that articulate criteria for ChatGPT texts. By creating such rubrics, students using this technology may notice that ChatGPT likely cannot produce the depth they are looking for and therefore would not be worth copying word for word. They rubrics may also prime them to engage certain critical thinking skills that they may not naturally bring to texts produced by generative AI. Because ChatGPT texts are initially impressive and appear relatively polished on the surface, encouraging a dive beneath the surface could be pedagogically beneficial.