Composing with AI

Large Language Model Applications for Style Pedagogy

Christopher Eisenhart University of Massachusetts, Dartmouth

Introduction

In this study, I test and discuss the potential for using Large Language Models (LLMs) when working with writing students who are studying style. Style curriculum and pedagogy is interestingly positioned in relation to intertext and context. While we all know that rhetorically effective writing always requires contextualization and attendant, sensitive revision, a composition curriculum such as Joseph Williams’ Style helps students develop the editing and revision skills of style typically by a) spending large amounts of time and effort on the grammar, syntax, and economy of individual sentences and paragraphs; and also by b) largely ignoring context or, more precisely, by presuming a pan-context of Standard Written English in academic, journalistic, or business writing. This situational decontextualization of the original text to provide students with an instructional context to work on the concepts and skills of revision and editing is an important step in helping students to develop these editing and revision perspectives and tools.

Presumably, these de-contextualized or context-fixed curricular moments might be perfect for maximizing the usefulness of LLMs and for helping students learn to use them. But how do LLMs perform in these tasks? Can LLMs problematize this curriculum by simply "doing" these exercises on their own, given the exercises’ relative lack of contextual specificity? As Vee has admonished us, "[T]eaching writing with LLMs ethically means understanding what aspects of writing they can’t do." In the tradition of testing software for composition pedagogy (from Smye 1988 to Knowles 2022), I have worked with the LLM ChatGPT 3.5 (CGPT) to complete Joseph Williams' curriculum from Style: Lessons in Clarity and Grace (12th edition, with William Bizup). The goal of this work has been not to criticize CGPT for what it can't do, but to determine what it can do as a potentially useful and inevitable tool for students doing a particular kind of editing and revision work. I also have worked with the goal to recreate what students might likely do with the tools and the curriculum, to anticipate its role in our shared experience. In what follows, I very briefly discuss the idea of LLMs as intertextual tools and what that conceptualization can do for us. I then also briefly outline Joseph Williams' style curriculum before I report my analysis of CGPT’s performance executing that curriculum.

LLMs as Intertextual tools

Charles Bazerman once described intertextuality in a metaphor similar to those used to describe Large Language Models (LLMs): "We create our texts out of the sea of former texts that surround us, the sea of language we live in. And we understand the texts of others out of that same sea" (83–84). The concept of intertextuality and our application of intertextuality to explain human writing also can be used to explain what LLMs do. LLMs respond to queries by providing word sequences, where the next word is chosen because of its tokenized and calculated relation to the other words in the pattern. In this way, LLMs can be thought of as engaging in horizontal intertextuality, where they calculate the next word based on its appropriateness to the string of words that have come before in a theoretically endless string of words to come, an endless dialog between this text and all text. Here I follow many others in borrowing the notion of horizontal intertextuality from the work of Julia Kristeva (1980), who discussed the ways texts could be horizontally intertextual, with their readers, but also with those texts which came before.

Texts can also be thought of as vertically intertextual, attaching to contexts and typologies of form sensitive to those contexts. While LLMs do horizontal intertextual work as they choose the next word in the sequence based on prior relationships calculated from among their massive bases, unless (and even perhaps when) prompted, they do not do the vertical intertextual work of drawing from rhetorical contexts. Description and scholarship around generative AI and composition emphasize that LLMs are not primarily contextual tools, and that the role of the human writer includes providing context and being sensitive to context. As Morrison (2023) pionts out, "while AI has the potential to be a valuable tool for writers, it is important to note that it is not a substitute for human creativity and critical thinking. Writers should still be mindful of the context and purpose of their writing, and should use AI tools as a supplement to their own knowledge and expertise, rather than relying on them entirely" (p. 157). So, we might assume, LLMs are good at horizontal intertext, but not so good at vertical intertext or context. As I show below, the style curriculum is based largely on decontextualized sentences, leading to the hypothesis that LLMs might perform well at the challenges of this particular curriculum. In sum, those results suggest that some of ChatGPT's passes at the curriculum were consistently successful, especially in terms of subject and verb identification and revisions for concision. In these cases, using CGPT would be especially useful to the course as an illustration to students who struggle with grammatical analysis of subjects and verbs in problematic original sentences, and could use CGPT as a tutor with some confidence. Over the course, where CGPT struggled tended to be where students struggled, relying on context clues and inference to provide the necessary components that might make a sentence more clear and efficient in Williams' terms. This does not negate the value of using CGPT in the course, as I discuss later, but rather shows how using CGPT will leave places for students and the class together to diagnose that output and continue to revise toward Williams' stylistic standards.

The framework of Joseph Williams' style curriculum

In this study, I use the 12th edition of Style, which was revised, edited, and updated by Joseph Bizup after Williams' death in 2008, and it should rightly be cited as Williams and Bizup. However, given the standing of Williams's curriculum through many years and editions, I refer to it as the Williams style curriculum in passing throughout, with profound gratitude that Prof. Bizup continues to keep the curriculum alive and lively for our students. Like so many others, I have found this curriculum to be incredibly useful for mid- to late-career undergraduate students interested in work that may include writing and editing. I have taught the curriculum now for over twenty years, first learning to do so under the generous tutelage of Erwin Steinberg as a doctoral student at Carnegie Mellon.

Bizup's introduction to this edition identifies three questions as its heart: "What is it in a sentence that makes readers judge it as they do? How do we analyze our own prose to analyze their judgments? How do we revise a sentence so that readers will think better of it?" (v). Most important to note here is the focus on the sentence. While later lessons to take on passages and paragraphs, and one lesson touches on ways works can achieve "global coherence" across texts, the fundamental focus and work of this curriculum is the diagnosis and revision of problems in sentences that may confuse or confound readers. Overall, students are meant to leave the curriculum prepared to recognize sentences that are too difficult for their intended audience to read, able to diagnose the problems in the sentence that cause those difficulties and prepared to revise the sentence on strategies toward a narrative sense of clarity and syntactic structure for managing complexity.

The curriculum proceeds from the argument that readers struggle least when sentences are active and narrative, and the starting point for every revision is to reassess the sentence in terms of a story it is endeavoring to tell. From there, we are instructed to revise sentences so that characters are its subjects and actions are its verbs, and all other guidance is built upon that foundation. When fully developed, that foundation bears the following conceptual framework:

Dispatching quickly with issues of Correctness, the bulk of the curriculum is organized into Lessons that build in complexity, depending on the early lessons to successfully complete the late. Students must immediately be able to identify subjects and verbs in sentences, and their control of those grammatical and syntactic foundations determines their success. These lessons are organized under concepts, starting with clarity and moving through cohesion, coherence, emphasis, concision and shape. In each case, these concepts are (helpfully) just names given to sets of diagnostic questions and strategies for revision more than they are language theory. While each lesson typically closes with an "exercise" to identify in one’s own writing the opportunities for revision therein, the value of each lesson is its instruction in diagnoses and revision and the carefully crafted sentences in its provided exercises that focus practice. Overwhelmingly, those sentences are removed from any particular context, other than what the reader may imagine.

Analysis: Working with ChatGPT (CGPT)

The transcripts of CGPT completing these exercises became the data for studying the LLMs' performance and approaches to these kinds of problems. Following the goal to anticipate students' potential experiences with CGPT and this curriculum, I did not seek to optimize CGPT through expert practices. For example, while I did often prompt CGPT to consider Williams' principles of style, I did not load and train CGPT on the Williams text because I doubted that our students would train CGPT in such a way. In sum, what follows suggests that students and teachers of style can usefully employ ChatGPT when analyzing and revising sentences using Williams' principles, although they cannot rely on CGPT to complete the curriculum successfully, aiding but not invalidating instruction and student revision.

Step One: Diagnosing Subjects and Verbs, Characters and Actions

Among the number of principles that guide the Williams style curriculum, some are explicitly grammatical and syntactic. The first steps for students in the curriculum are to diagnose sentences to identify problems in those sentences. The first of those diagnostics is to identify the subject and verb in a sentence, the foundation upon which the rest of Williams' guidance builds. While identifying subjects and verbs in sentences is foundational, Williams actually spends little time providing curriculum and exercises for doing so, leaving instructors like me the freedom to invent exercises for students to practice this sometimes dusty and rusty skill. My first task with ChatGPT (CGPT), then, was to employ some of these practice sentences and ask CGPT to identify subjects and verbs therein. Here are a few examples from the 17 I used to test CGPT:

A course in basket-weaving does count for credit under the Creative Activities requirement.
After sunset astronomers will be able to see the new comet. Just one of these new super high density disks can hold the entire manuscript.
If the electric fence were down, the dinosaurs would be able to reach the compound.

The sentences range from simple to complex and difficult, including dense noun phrases that distanced the subjects from the verbs, passive verb constructions, and an "it-clause" sentence. Overwhelmingly, CGPT correctly identified subjects and verbs in the sentences. Its single error came in the sentence: "Just one of these new super-high density disks can hold the entire manuscript." CGPT identified "disk" as the subject and "can hold" as the verb. Most students do too, until re-trained and practiced. Technically, "one" is the subject of the sentence with disk as the noun of the modifying prepositional phrase. It’s also unknown why CGPT made the subject singular, especially since either "disks" or "disk" would agree with the verb "can hold." In sum, CGPT gets 16 of 17 correctly, outscoring most students in the early days of the term on this introductory activity.

After subject-verb identification, the next diagnostic activity of the early Williams curriculum is to identify characters and actions within a sentence and to identify whether or not they are the subjects and verbs of that sentence. Williams teaches that an active, narrative style is most readable for most audiences in most contexts. Early tasks involve identifying the potential characters and their actions in sentences, and whether or not they are the subjects and verbs of those sentences. This is a necessarily interpretive activity, as there may be more than one potential character within a sentence. Students are required to decide who the sentence should be about, who should be the character in the sentence taking the actions that are the verb (if the sentence is transitive/transactive). Students also have to identify an action in a sentence where—in the original—that action might be nominalized or implicit. Identifying character and action are both more challenging interpretive tasks for most students than identifying grammatical subjects and verbs.

ChatGPT did well again with this diagnostic activity. I gave it several sentences and the task of identifying the subject, verb, character, and action of the sentence. CGPT identified all subjects and verbs correctly. It also identified useful characters in all sentences. CGPT struggled more with identifying the action within the sentence. For example, in the sentence:

The blueprint for the proposed, blue house has been a challenge for the architects beyond the expectations that they brought to the project.

ChatGPT identified "challenge" as the action to go with the character architects, but our target revision would not be that "architects challenge." Like many students, CGPT is looking for the action to be explicit within the sentence when, in this case, to make the sentence active with characters as the subjects the verb will need to be provided from context. However, when I asked CGPT a second time, it provided a successful answer: characters-architects, actions-faced. This would allow the revision "Architects faced greater challenges than they expected when they designed the blueprint for the proposed blue house." This would be a successful, target revision for the Williams curriculum. So, on repeat tries, CGPT went from attempting to find the action from within the sentence to finding an acceptable action that wasn’t.

Step Two: Revising Sentences Based on the Diagnostics

The Williams curriculum scaffolds from diagnostic to revision by requiring students to continue to explicitly identify subjects and verbs, characters and actions prior to revising. The hope is that students will continue their focus on making the sentences more active as their first of ultimately several sequential steps in revising toward more readable sentences. My prompt and CGPT’s response for the first sentence:

Good morning. Using Joseph Williams principles of Style, identify the subjects verbs characters and actions in the following sentence: There has been speculation by educators about the role of the family in improving educational achievement.

ChatGPT
Good morning! To identify the subjects, verbs, characters, and actions in the sentence using Joseph Williams' principles of style, we can break it down as follows:

1. Subject: "There" - This is an expletive subject that doesn't refer to a specific character but serves as a placeholder.
2. Verb: "has been speculation" - This is the verb phrase. "Has been" is the auxiliary verb, and "speculation" is the main verb.
3. Characters:
   • "educators" - This refers to the group of people who are engaging in speculation.
   • "the family" - This refers to another entity discussed in the sentence.
4. Actions:
   • "speculation" - This is the action taken by educators.
   • "improving" - This is the action or purpose related to the family, which is improving educational achievement.

So, in summary:

• Subject: "There"
• Verb: "has been speculation"
• Characters: Educators and the family
• Actions: Educators engage in speculation, and the family is involved in improving educational achievement.

In this five-sentence exercise, CGPT identified correctly subjects and verbs; however, where in a previous exercise it had identified individual subjects and verbs, this time it returned subject and verb phrases. In this example, CGPT identified the verb phrase with part of the predicate that is not the verb, the nominalization “speculation” in “has been speculation.” In this sentence, “has been” is a linking verb to connect with the placeholder subject “There.” “Speculation” is a nominalization that acts as a subject complement. Again, if a student identified this verb as “has been speculation,” we would need to coach them toward more precise revision, to ensure they could diagnose the nominalized action as distinct from the verb. When diagnosing subjects, CGPT also sometimes identified phrases, such as “Attempts by economists,” rather than simply “Attempt.” For another sentence, it identified the subject as, “The loss of market share to Japan by domestic automakers.” Working with a student, I would have to ask for further refinement, because the objective of the exercise—identifying the individual, grammatical subject to be able to determine if it is the character—would not have been met.

When identifying characters and actions, CGPT once again demonstrated a tendency similar to students to limit their search to words from the original sentence rather than supply an implicit character which the exercise sometime requires. For example, it identified “hundreds of thousands of jobs” as a character, rather than provide the implicit “workers,” and in another sentence identified “achievement” as a potential action in a sentence about education, but did not provide the character “students.” Further, in an academic, passive sentence, CGPT identified no possible character. From the original sentence, “There is a need for an analysis of library use to provide a reliable base for the projection of necessary resources,” Williams’ guidance would be to provide an authorial “we” as the character, but CGPT returned, “Characters: Not specified (general need).” CGPT’s tendency to not identify implied characters from actions highlights the interpretive work required in this diagnostic task of the Williams curriculum.

Having diagnosed for subject and verb, character and action, Williams guides revisions to the sentences for clarity, prompting students to revise with an identified character as the subject and an action as the verb. CGPT returned the following revisions:

1. "Educators have been speculating about the role of the family in enhancing educational achievement."
2. "Customers grew frustrated due to the store's price increases."
3. "Economists have struggled in their attempts to define full employment."
4. "Domestic automakers lost market share to Japan, causing the disappearance of hundreds of thousands of jobs."
5. "An analysis of library usage is necessary for us to establish a reliable foundation for resource planning."

Sentences 1–4 all complete the basic task, with subjects and verbs revised to characters and action. In a class discussion, we might request that students examine whether sentences would be improved with further revisions around achievement, increases, and disappearance (loss), but all 4 would demonstrate having accomplished the objective of the exercise. Sentence 5 fails, as CGPT did not diagnose a character and does not provide one. Interestingly, CGPT did introduce a character in this revision: “us” is absent in the original, but CGPT didn’t make that character the subject of the sentence. We might judge these revisions as generally successful, but with room for improvement, as would be typical of students at this stage.

According to the Williams curriculum, sentences without characters and actions as subjects and verbs often have two kinds of common grammatical and stylistic problems: nominalizations and passive verb constructions. A number of Williams’ exercises ask students to pay attention to these types of problematic constructions and think of them as the sites to find “missing” characters and actions in their revisions.

When given the guidance to revise sentences by making nominalizations into actions and verbs, and to make characters into subjects, CGPT performed fairly well, especially where nominalizations had evident characters. For example, CGPT revised “Lincoln's hope was for the preservation of the Union without war, but the South's attack on the Fort Sumter made war an inevitability” into “Lincoln hoped to preserve the Union without war, but the South's attack on Fort Sumter made war inevitable.” This improvement would meet the threshold for success with a student, because the main, root subject and verb are revised. However, CGPT missed the opportunity to revise the second clause of the sentence, leaving the nominalized “attack” as the subject. A more thoroughgoing revision would have perhaps been, “…but when the South attacked Fort Sumpter, they made.” We would encourage the student to revise for character and action consistently, not necessarily just the first occurrence.

In sentences where a character was not explicit in the original sentence, CGPT often failed to provide one in the target sentence when revising nominalizations. For example, the original sentence provided to ChatGPT, “Contradictions among the data require an explanation” was revised to, “An explanation is required for the contradictions among the data.” This is no substantive improvement, where Williams is looking for a target revision with character, such as “We must explain…”. This example and more like it in the exercises solving for nominalizations further demonstrate CGPT’s reticence to provide subjects/characters that are not explicit in the original. If I further prompted to revise a specific sentence and I supplied a possible character to use as subject, CGPT revised effectively. However, this isn’t something most, or perhaps any students would do. Identifying an absent or implied character is the difficult interpretive work here for students as well; if they would have to provide that interpretation, CGPT fails to provide much added value.

Similarly, because passive constructions also tend to displace or hide active characters, CGPT failed frequently at providing characters as subjects when revising sentences with passive constructions. The following are representative samples:

CE: Please revise the following sentences so that characters are the subjects and actions are the verbs: A decision about forcibly administering medication in an emergency room setting despite the inability of an irrational patient to provide legal consent is usually an on-scene medical decision. The performance of the play was marked by enthusiasm, but there was a lack of intelligent staging. Despite the critical panning of the show's latest season the love of the loyal fans was not affected. The rejection of the proposal was a disappointment but not a surprise because our expectation was that a political decision had been made.

ChatGPT
Certainly, here are the revised sentences with characters as the subjects and actions as the verbs:

1. Medical professionals usually make a decision about forcibly administering medication in an emergency room setting despite the inability of an irrational patient to provide legal consent.
2. The actors' performance of the play was marked by enthusiasm, but the director lacked intelligent staging.
3. Despite the critical panning of the show's latest season, the loyal fans' love was not affected.
4. The rejection of the proposal disappointed us, but it did not surprise us because we expected a political decision had been made.

Across many sentences with passive constructions, CGPT failed to consistently identify and provide a character to employ as a subject of an action/verb. In revision 1 above, CGPT does infer the character/subject “medical professionals” and provides “make” as an action verb. However, in 2 we see that while CGPT did identify actors as an implied subject, it did not revise to make it active (such as “actors performed enthusiastically”). CGPT struggled in these two ways: identifying an appropriate, implied character and consistently converting appropriate, nominalized actions into verbs.

Large Language Model Applications for Style Pedagogy

Christopher Eisenhart

Step Three: Revising for Cohesion, Coherence, Emphasis, Concision, and Shape

The diagnostics and revision of individual sentences to this point in the Williams curriculum depends upon the following layers of understanding:

In the next steps of the curriculum, the conceptual complexity increases, calling on students to understand and apply:

Taken in turn, the curriculum next asks students to graduate from revising sentences to revising passages and paragraphs, and then to diagnose and revise for cohesion and coherence. Williams definitions of these concepts amount to:

Cohesion—the reader’s recognition of a topical connection between any two sentences, particularly in how one sentence ends and the next begins
Coherence—the reader’s recognition of topical connection across an entire passage/paragraph, particularly in how sentence subjects are consistent or related topics

Attention to these topics requires attention to the syntax of sentences beyond previous diagnostics. To describe sentences in terms of sentence topics and comments, students must identify not only subject and verb but think also in terms of subject and predicate. To think in terms of given/old and new information, students have to think outside the individual sentence, either to context or to the previous sentence(s). In all of these tasks, the work is decreasingly grammatical, but remains syntactic and is increasingly context/invention dependent.

The Williams curriculum teaches managing cohesion and coherence by managing what information is given and new, encouraging given (familiar) information to appear in the subject/topic of the sentence and new information to appear in the predicate/comment of the sentence. In the exercises, Williams asks students to revise so that “old information comes first in the sentence” and to revise “to give sentences more consistent topic strings.” ChatGPT (CGPT) improved all passages on that guidance, but did not succeed in every sentence, and returned for the first time in these exercises a non-sensical sentence. Further, CGPT did not simultaneously revise for subject/verb as character/action, which we would hope students would carry forward with them from previous lessons.

When prompted to start sentences with given information, CGPT sometimes identified appropriate topics to use as the beginnings and subjects of a sentence to improve given—new information flow, but not consistently. In one revision—a metadiscursive passage—CGPT started by appropriately supplying an implied authorial “we” as the subject/character/topic: “In our report, we will highlight the components of Abco's profitability, with a particular emphasis on its advantages versus competitors, especially the growth in Asian markets.” However, that subject/topic was lost in subsequent sentences. When prompted to revise to begin with consistent topic strings, CGPT did not show evidence of being able to identify what a consistent topic was unless prompted. For example, the following revision:

Vegetation covers the earth, except for those areas continuously covered with ice or utterly scorched by continual heat. Richly fertilized plains and river valleys are situated in regions where plants grow most abundantly, as well as at the edge of perpetual snow in high mountains. The ocean and its edges, along with lakes and swamps, exhibit dense vegetation. Even in the cracks of busy city sidewalks and seemingly barren cliffs, plants thrive. Before humans existed, the earth was cloaked in vegetation, and it will continue to be so long after evolutionary history has passed us by.

Here, the revised subjects alternate from vegetation to earth and topographical settings, which does not demonstrate Williams’ objective of controlling for consistent topics. Interestingly, when I marked each instance of vegetation/plants with parentheses, and asked CGPT to revise to make those the topics, it attempted to do so, and succeeded in three sentences, but also provided the problematic: “In regions where (plants) grow most abundantly, richly fertilized plains and river valleys are situated, as well as at the edge of perpetual snow in high mountains.” This would seem to suggest that CGPT was unable to categorize topics as necessarily connected to the subject and verb of the root, independent clause of the sentences. It also was one of two nearly non-sensical revisions in these exercises that CGPT provided, which students would be unlikely to provide as workable revisions.

Emphasis

The Williams curriculum shifts focus from the subjects/topics of sentences to their predicates/comments when creating emphasis. The principle of emphasis pairs with coherence. There must be consistent topics for a passage or paragraph to cohere. Where we have variation is in the comments or predicates of sentences, conveying new information. Further, in each new comment/predicate, there is often a most important or main new idea. Positioning that idea at the end of the sentence gives it the best emphasis. Exercises on emphasis ask students to control topics (find “appropriate” topics where only single sentences are available) and to emphasize the most important information by positioning it at the end of the predicate.

ChatGPT demonstrated several of Williams’ strategies in these revisions for emphasis:

Using the principles of Williams' book Style, please revise each of the following sentences for clarity and to emphasize the most important words: The president's tendency to rewrite the Constitution is the biggest danger to America, in my opinion, at least. A new political philosophy that could affect our society well into the twenty-first century may emerge from these studies. There are limited opportunities for faculty to work with individual students in large colleges and universities. Building suburban housing developments in floodplains has led to the existence of extensive and widespread flooding and economic disaster in parts of our country in recent years, it is now clear. The teacher who makes an assignment of a long final term paper at the end of the semester and who then gives only a grade and nothing else such as a critical comment is a common object of complaint among students at the college level. Renting textbooks rather than buying them for basic required courses such as mathematics, foreign languages, and English, whose textbooks do not go through yearly changes, is feasible, however, economically speaking.

ChatGPT
Applying the principles from Williams' book "Style," I've revised the sentences to enhance clarity and emphasize key points:

1. **In my opinion, the most significant danger to America is the president's inclination to continually amend the Constitution.
2. From these studies, a new political philosophy may emerge, shaping our society well into the twenty-first century.
3. In large colleges and universities, there are few opportunities for faculty to engage individually with students.
4. It is now evident that constructing suburban housing developments in floodplains has caused extensive, widespread flooding, and economic disasters in recent years.
5. Among college-level students, a common source of complaint is the teacher who assigns a lengthy final term paper at the end of the semester, offering only a grade without any critical feedback.
6. Economically speaking, it is feasible to rent textbooks for fundamental required courses such as mathematics, foreign languages, and English, as these textbooks do not undergo yearly revisions.

Focused on the emphasis position at the ends of predicates, we see that most sentences end with ideas that are arguably the main point to be emphasized. Exceptions are probably sentences 4 and 6, where 4 ends with contextual/timeframe information (in recent years) and 6 ends with a qualifying phrase rather than the main point. All sentences demonstrate revisions that employ the syntactic strategies Williams suggests for managing sentence position, such as the pre-core or contextual phrases before the subject in 1, 2, 5, and 6, and the “there are” and “it is” shifts in sentences 3 and 4.

CGPT’s revisions to whole passages were somewhat more problematic, but also point at the challenge of choosing what is a main idea to be emphasized, a challenge Williams repeatedly acknowledges in the curriculum. For example, CGPT provided the following revision:

CE: Please revise the following passage so that sentences begin with appropriate topics and end with appropriate emphasis: Athen's catastrophic Sicilian Invasion is the most important event in Thucydides' History of the Peloponnesian War. Three quarters of the history is devoted to setting up the invasion because of this. Through the step-by-step decline in Athenian society that Thucydides describes, we can see how he chose to anticipate the Sicilian Invasion. The inevitability that we associate with the tragic drama is the basic reason for the need to anticipate the invasion.

ChatGPT
Certainly, here's the passage with sentences starting with the appropriate topics and ending with appropriate emphasis:

Thucydides' History of the Peloponnesian War begins with the most significant event, Athens' catastrophic Sicilian Invasion.

Setting up the invasion occupies three quarters of the history.

Through Thucydides' detailed portrayal of the step-by-step decline in Athenian society, we gain insight into why he chose to anticipate the Sicilian Invasion.

The need to anticipate the invasion is rooted in the inevitability we associate with a tragic drama, which serves as its fundamental reason.

CGPT mistakenly places the Sicilian Invasion at the beginning of the text in the first sentence and also confuses the relationship of anticipation and tragedy in the final sentence. However, the larger problem with the revision is that CGPT never makes Thucydides the character and the topic of the revised sentences. By not having consistent topic strings, sentences are not coherent enough to have an opportunity at creating appropriate emphasis. It appears that CGPT is unable to successfully read the awkward original and draw from it correct revisions or grammar and syntax choices that correspond with Williams’ guidance on coherence and emphasis across whole passages.

Concision

Williams’ curriculum on concision targets sentence features such as metadiscourse, repetitions, and negation for students to diagnose and remove. Further, Williams demonstrates that sentences written in active voice (again, characters/subject and actions/verbs) tend to be more efficient than their nominalized or passive alternatives. Most of the concision curriculum then practices the basic revision steps paired with new diagnoses for non-essential phrasing.

Overall, CGPT does very well revising for concision. In most but not all exercises, CGPT eliminated redundant phrases. It fares perhaps less well identifying what might be unnecessary phrases, such as metadiscursives, and hedging and stance markers (e.g. the first is, the second is; or I believe, we can assume). Further, CGPT demonstrated the ability to revise negatives to positives successfully, and in one exercise correctly diagnosed a double-negative as the reason one sentence was clearer than another. Notably, even when prompted to employ Williams’ principles in revision, CGPT does not consistently provide the concision revisions with as active a revision in character/subject and action/verb as would be possible.

Sentence Shape

Shape is both Williams’ most summative concept for style and also that which is least concretely defined. He does provide prescriptions that we can take as adding up to a description (paraphrased). A shapely sentence:

1. Gets quickly to the subject, and then quickly to the verb and its object. It positions the subject and verb close together.
2. Gets to the point at the start in a short, main clause that states the claim.
3. After the main clause, it avoids adding a string of additional subordinate clauses.

Presumably, a shapely sentence also employs the efficiency and concision found in active sentences that are again written with subjects as characters and actions as verbs, and the diagnostics in the lesson employ that diagnostic and revision step. While these descriptions are developed in grammatical and syntactic terms, we can note that the challenges of identifying characters and actions, main ideas, and old and new information for emphasis still remain.

The following revisions from CGPT illustrate the challenges. Note, CGPT had already been instructed to follow Williams’ curriculum on shape earlier in this chat:

Please revise the following sentences for concision by eliminating wordiness and correcting interruptions: 1. TV "reality" shows, because they have an appeal to our fascination with real-life conflict because of our voyeuristic impulses, are about the most popular shows that are regularly scheduled to appear on TV. 2. Insistence that there is no proof by scientific means of a causal link between tobacco consumption and various disease entities such as cardiac heart diseases and malignant growth, despite the fact that there is a strong statistical correlation between smoking behavior and such diseases, is no longer the officially stated position of cigarette companies. 3. The continued and unabated emission of carbon dioxide gas into the atmosphere, unless there is a marked reduction, will eventually result in serious changes in the climate of the world as we know it today.

ChatGPT
Certainly, here are the revised sentences with wordiness eliminated and interruptions corrected:

1. TV "reality" shows, because they tap into our fascination with real-life conflicts and voyeuristic impulses, rank among the most popular scheduled TV programs.
2. Cigarette companies no longer officially maintain that there is no scientific proof of a causal link between tobacco consumption and diseases like cardiac heart diseases and malignant growth, despite a strong statistical correlation between smoking behavior and such conditions.
3. Unless there is a significant reduction, the continued and unmitigated emission of carbon dioxide into the atmosphere will inevitably lead to significant changes in today's world climate.

In sentence 1, CGPT appears to fail to diagnose the subordinate clause mid-subject as an interruption, as Williams would have students do. The interruption further confuses the new information about reality shows, (reads as if we are fascinated with conflicts and impulses, rather than our fascination and our impulses). The subject remains far and densely divided from the verb. In sentence 2, the subject and verb are immediate and active; however, the double-negative that confounds concision in the original remains in CGPT’s revision. Further, CGPT has confused the effect of a strong statistical correlation in this revision. Sentence 3, however, is a significant improvement over the original, with a helpful pre-core phrase and an appropriate emphasis in the predicate. There was perhaps an opportunity for a more active version that we would like to see from students, (“Unless we reduce our emissions of carbon dioxide into the atmosphere, it will lead to…”). Still, in CGPT’s sentence 3 we do see revisions improving shape along the lines Williams proposes. On balance, however, the decisions required for shape appear too many and too complex for CGPT to deliver consistently, correctly, and sensibly.

Discussion

Implicit in any study of computers and writing pedagogy must be the question, “What will happen when my students use this tool to complete their work?” Setting aside the ethical and policy aspects of this question, this study attempts to answer in the most pragmatic and descriptive way possible. Any student could interact with ChatGPT (CGPT) basically as I've done here, asking it to complete the exercises in Williams' curriculum, without having trained CGPT or taken other, more expert preparations. What we see in the study above is the kind of results they'll mostly encounter (not exactly of course; responses will change from instance to instance). Overall, CGPT improves sentences in its revisions from the original versions. It is mostly successful at some tasks of diagnosis and revision. However, it also leaves problems in its revisions which require expertise to then diagnose and revise. Students who input the exercises into CGPT will only occasionally receive results that match the objectives of the curriculum. That CGPT’s failures tend to be those where interpretation, invention, and contextualization is required is not a surprise.

Interestingly, these are the same moments that students tend to struggle with, until they realize that the solutions to the stylistic problems they're finding often do not exist within the original sentence (e.g. finding appropriate characters, deciding what is a main idea). Fortunately, while CGPT’s horizontally intertextual focus may not improve on this point, students do. Using CGPT with students would present an opportunity to demonstrate limitations of these types, and promote discussions of possible remedies, as we do with student attempts in workshops together.

One potential explanation for CGPT's limitations also resides in its horizontal intertextuality and lack of contextual awareness. Being intertextual in this way, tending to reproduce what it has found to previously be the most statistically likely output should create a descriptive bias rather than a prescriptive one. In the same ways that critics have discussed the bias of LLMs based on the biases of their corpuses (cf. Byrd, 2023), CGPT will revise sentences toward what other sentences have mostly tended to do, rather than what is “best” to do according to any particular stylistic paradigm, such as Williams'. Most sentences in the corpuses on which CGPT trains probably aren't clear, cohesive, coherent, concise, and shapely. Successive and amended prompts can improve CGPT's revisions, but its default is going to be to do what is descriptive of what people mostly do, not following the prescriptions of the curriculum.

This then leaves the pedagogical potential. How might using ChatGPT create or further opportunities for students' learning and my teaching of written style? When I'm teaching this curriculum, I use Williams' exercises as a jumping off point for students. Students read each Lesson and attempt to complete its exercises with minimal demonstration from me. This is low-stakes work that I've found makes them well and truly invested when I then throw their answers up on the board for everyone to see and discuss what they've improved in their revision and what is left to be done. Then we finish those revisions, live and together. I follow up with additional sentences, crafted to represent the problems Williams has given us to work on together in the classroom, and typically find the students almost competitively engaged in the game of revision. As a class, we might use LLMs in much the way we use the work of other students, to learn these skills of diagnosing and revising the problems in first revisions of problematic sentences. While the greatest impact may come from workshopping student work, asking CGPT to provide initial diagnoses and revisions together with the class can allow us to lower the stakes of this practice even further, making CGPT a helpful baseline and stand-in for initial drafts of this revision work. When we show CGPT doing the exercises, the challenging work of diagnosing and improving these exercises will still need to be done.

CGPT seems to be best at subject-verb identification and revision and the work of revising for concision. For that percentage of my students who struggle with subject/verb identification, I'd welcome practice with the tool that might then help them begin to see the patterns that identify subjects and verbs when their classmates and I have largely moved on. When it comes to the tasks of finding the narrative implicit in sentences to serve as the basis for inventing characters and actions, determining what is given and what is new information, identifying main ideas to position them for emphasis, CGPT tends to struggle with what students struggle with. Revisions to style are not purely intra-textual activities, and the solutions to the problems in bad sentences are not often visible, evident, or explicit. We can then use CGPT together to stand in for initial draft work, to demonstrate steps toward diagnosis and revision that leave work for us to practice together.

References