Composing with AI

Large Language Model Applications for Style Pedagogy

Christopher Eisenhart University of Massachusetts, Dartmouth

Introduction

In this study, I test and discuss the potential for using Large Language Models (LLMs) when working with writing students who are studying style. Style curriculum and pedagogy is interestingly positioned in relation to intertext and context. While we all know that rhetorically effective writing always requires contextualization and attendant, sensitive revision, composition curriculum such as Joseph Williams’ Style helps students develop the editing and revision skills of style typically by a) spending large amounts of time and effort on the grammar, syntax, and economy of individual sentences and paragraphs; and also by b) largely ignoring context or, more precisely, by presuming a pan-context of Standard Written English in academic, journalistic, or business writing. This situational decontextualization of the original text to provide students with an instructional context to work on the concepts and skills of revision and editing is an important step in helping students to develop these editing and revision perspectives and tools.

Presumably, these de-contextualized or context-fixed curricular moments might be perfect for maximizing the usefulness of LLMs and for helping students learn to use them. But how do LLMs perform in these tasks? Can LLMs problematize this curriculum by simply “doing” these exercises on their own, given the exercises’ relative lack of contextual specificity? As Vee has admonished us, “[T]eaching writing with LLMs ethically means understanding what aspects of writing they can’t do.” In the tradition of testing software for composition pedagogy (from Smye 1988 to Knowles 2022), I have worked with the LLM ChatGPT 3.5 (CGPT) to complete Joseph Williams' curriculum from Style: Lessons in Clarity and Grace (12th edition, with William Bizup). The goal of this work has been not to criticize CGPT for what it can't do, but to determine what it can do as a potentially useful and inevitable tool for students doing a particular kind of editing and revision work. I also have worked with the goal to recreate what students might likely do with the tools and the curriculum, to anticipate its role in our shared experience. In what follows, I very briefly discuss the idea of LLMs as intertextual tools and what that conceptualization can do for us. I then also briefly outline Joseph Williams' style curriculum before I then report my analysis of CGPT’s performance executing that curriculum.

LLMs as Intertextual tools

Charles Bazerman once described intertextuality in a metaphor similar to those used to describe Large Language Models (LLMs): “We create our texts out of the sea of former texts that surround us, the sea of language we live in. And we understand the texts of others out of that same sea” (83–84). The concept of intertextuality and our application of intertextuality to explain human writing also can be used to explain what LLMs do. LLMs respond to queries by providing word sequences, where the next word is chosen because of its tokenized and calculated relation to the other words in the pattern. In this way, LLMs can be thought of as engaging in horizontal intertextuality, where they calculate the next word based on its appropriateness to the string of words that have come before in a theoretically endless string of words to come, an endless dialog between this text and all text. Here I follow many others in borrowing the notion of horizontal intertextuality from the work of Julia Kristeva (1980), who discussed the ways texts could be horizontally intertextual, with their readers, but also with those texts which came before.

Texts can also be thought of as vertically intertextual, attaching to contexts and typologies of form sensitive to those contexts. While LLMs do horizontal intertextual work as they choose the next word in the sequence based on prior relationships calculated from among their massive bases, unless (and even perhaps when) prompted, they do not do the vertical intertextual work of drawing from rhetorical contexts. Description and scholarship around generative AI and composition emphasize that LLMs are not primarily contextual tools, and that the role of the human writer includes providing context and being sensitive to context. “While AI has the potential to be a valuable tool for writers, it is important to note that it is not a substitute for human creativity and critical thinking. Writers should still be mindful of the context and purpose of their writing, and should use AI tools as a supplement to their own knowledge and expertise, rather than relying on them entirely.”’ (Morrison 2023). So, we might assume, LLMs are good at horizontal intertext, but not so good at vertical intertext or context. As I show below, the style curriculum is based largely in decontextualized sentences, leading to the hypothesis that LLMs might perform well at the challenges of this particular curriculum. In sum, those results suggest that some of ChatGPT's passes at the curriculum were consistently successful, especially in terms of subject and verb identification and revisions for concision. In these cases, using CGPT would be especially useful to the course as an illustration to students who struggle with grammatical analysis of subjects and verbs in problematic original sentences, and could use CGPT as a tutor with some confidence. Over the course, where CGPT struggled tended to be where students struggled, relying on context clues and inference to provide the necessary components that might make a sentence more clear and efficient in Williams' terms. This does not negate the value of using CGPT in the course, as I discuss later, but rather shows how using CGPT will leave places for students and the class together to diagnose that output and continue to revise toward Williams' stylistic standards.

The framework of Joseph Williams' style curriculum

In this study, I use the 12th edition of Style, which was revised, edited, and updated by Joseph Bizup after Williams' death in 2008, and it should rightly be cited as Williams and Bizup. However, given the standing of Williams's curriculum through many years and editions, I refer to it as the Williams style curriculum in passing throughout, with profound gratitude that Prof. Bizup continues to keep the curriculum alive and lively for our students. Like so many others, I have found this curriculum to be incredibly useful for mid- to late-career undergraduate students interested in work that may include writing and editing. I have taught the curriculum now for over twenty years, first learning to do so under the generous tutelage of Erwin Steinberg as a doctoral student at Carnegie Mellon.

Bizup's introduction to this edition identifies three questions as its heart: “What is it in a sentence that makes readers judge it as they do? How do we analyze our own prose to analyze their judgments? How do we revise a sentence so that readers will think better of it?”(v). Most important to note here is the focus on the sentence. While later lessons to take on passages and paragraphs, and one lesson touches on ways works can achieve “global coherence” across texts, the fundamental focus and work of this curriculum is the diagnosis and revision of problems in sentences that may confuse or confound readers. Overall, students are meant to leave the curriculum prepared to recognize sentences that are too difficult for their intended audience to read, able to diagnose the problems in the sentence that cause those difficulties and prepared to revise the sentence on strategies toward a narrative sense of clarity and syntactic structure for managing complexity.

The curriculum proceeds from the argument that readers struggle least when sentences are active and narrative, and the starting point for every revision is to reassess the sentence in terms of a story it is endeavoring to tell. From there, we are instructed to revise sentences so that characters are its subjects and actions are its verbs, and all other guidance is built upon that foundation. When fully developed, that foundation bears the following conceptual framework:

Dispatching quickly issues of Correctness, the bulk of the curriculum is organized into Lessons that build in complexity, depending on the early lessons to successfully complete the late. Students must immediately be able to identify subjects and verbs in sentences, and their control of those grammatical and syntactic foundations determines their success. These lessons are organized under concepts starting with clarity and moving through cohesion, coherence, emphasis, concision and shape. In each case, these concepts are (helpfully) just names given to sets of diagnostic questions and strategies for revision more than they are language theory. While each lesson typically closes with an “exercise” to identify in one’s own writing the opportunities for revision therein, the value of each lesson is its instruction in diagnoses and revision and the carefully crafted sentences in its provided exercises that focus practice. Overwhelmingly, those sentences are removed from any particular context, other than that the reader imagines.

Analysis: Working with Chat GPT

The transcripts of CGPT completing these exercises became the data for studying the LLMs' performance and approaches to these kinds of problems. Following the goal to anticipate student's potential experiences with CGPT and this curriculum, I did not seek to optimize CGPT through expert practices. For example, while I did often prompt CGPT to consider Williams' principles of style, I did not load and train CGPT on the Williams text because I doubted that our students would train CGPT in such a way. In sum, what follows suggests that students and teachers of style can usefully employ Chat GPT (CGPT) when analyzing and revising sentences using Williams' principles, although they cannot rely on CGPT to complete the curriculum successfully, aiding but not invalidating instruction and student revision.