Composing with AI

Tell Me without Telling Me

LLMs in their current incarnation are ever attentive to modeling. Early in the release of ChatGPT, users were recounting stories of the LLM responding “better,” when they made their requests more politely. [cite] From that experience, they inferred that the bot must like that polite treatment. What was more likely happening (because we cannot be completely sure of what occurs inside that black box) was the LLM following the model of their “polite request,” and further that “politeness” described a careful use of language. Just as they input carefully chosen English, the bot outputs carefully chosen English. (Let us for the moment table the fact that there are implications about white supremacy and socio-economic status baked into these presumptions of politeness.)

From our discussions of those accounts, Douglass and I have developed “Tell Me without Telling Me” named after a contemporary meme on social media in which those who post essentially follow the age-old dictum “show don’t tell.” In this exercise, students try to get the LLM to produce writing using a certain diction or style, not by requesting it by performing a model of that style in their request. While this game is fairly innocuous when the style of writing is conventional, such as writing like a pirate, it can go into controversial territory easily, such as when students use slang from particular subcultures. With care, the exercise can open up a conversation about the nature of the LLM’s training model and how much exposure it had to certain types of writing. Without that care, the exercise could lead to performances of stereotypes. That did not happen with my students, however.