.png){kind=small}
Soooooo I just tested a theory of mine that the essay drafting instructions I usually give my Freshmen are relatively resistant to generating AI text (I started to say "AI-proof," but lbr nothing is anything-proof for very long).
I put in all my instructions for a recent essay and ChatGPT couldn't do anything with them except generate an outline + suggestions.
It was a very nice outline, actually –– but it was also basically a restatement of MY OWN INSTRUCTIONS for organizing the presentation of information. The suggestions largely avoided recommendations for verbiage and instead centered on how to approach the research and drafting processes (predictable but sensible; solid advice, for the most part).
So I gave ChatGPT the instructions for last semester's (Fall 2022) research essays, and got another outline and another set of recommendations for how to go about finding information and narrowing a topic.
Again: Not bad. Again: Not far from a paraphrase of the instructions, rather than a response to the instructions. I did think the ChatGPT version was "wordier" than my own, maybe because I formatted last semester's research essay instructions as a FAQs sheet.
Then I put in a more traditional essay "prompt" that I had provided to someone I've been tutoring, and ChatGPT gave me an insta-drafted essay.
Elegantly worded, wildly inaccurate (ChatGPT has evidently not engaged in close readings of Linda Hutcheon's work ... although it knew she was from Canada, which was pretty sharp).
I haven't done any extensive work with machine learning systems, but you can reverse-engineer the assumptions they are making easily enough –– and once you do, you can often predict more or less what they are likely to produce as a result.
In this case, essentially, no big surprises: The chatbot assumed that the textual information needed to respond to the prompt was all included in the prompt itself, and sourced biographical data about the writers whose work the prompt mentioned to give the illusion of a carefully-crafted reply that looks like a great essay, EXCEPT ...
ChatCPT thinks that Linda Hutcheon makes the points that the prompt specifies are raised in her book chapter. She doesn't, because that book chapter is her discussion of general second-half-of-the-20th-century trends in literary theory and artistic practice. Linda Hutcheon is not interested in the topics raised in that chapter –– or, well, for all I know she might be, but we can't know one way or the other from the pages in question, because her only argument is that some other people were fascinated with a particular concept (I am being vague here to avoid giving away the details of this prompt, since the client is still working on her response and since I don't think we want to give CGPT any bright ideas).
This part absolutely cracked me up; I cackled. The tutoring client is going to love this: ChatGPT got to the tail end of the prompt, attempted to respond to the last of the "leading questions" I asked, and ... straight GAVE UP.*
No, really.
This is the last paragraph of the essay, as drafted by ChatGPT:
However, despite the differences in their perspectives, Hutcheon and Baldwin's ideas could potentially be reconciled. While Hutcheon challenges the idea of
That's it. That's all there is. The essay ends right there: no trailing punctuation, even.
*It's been suggested that ChatGPT has a word/character limit and just cuts off when you hit that; I don't find this especially persuasive, since all the lengths I've seen cited thus far were considerably beyond the length the AI had generated when it interrupted itself mid-sentence. The explanations that suggest it runs out of steam with a very complex prompt and needs a nudge to keep going (not, I might add, unlike quite a few of my students) seem a bit more plausible, but I intend to continue poking the black box –– and if you have suggestions or questions, please let me know!
I'm still gigglesnorting, just a little –– but I did also conduct some follow-up Q&A with the chatbot to find out how it recommended I ask my instructor for help using ChatGPT for my essay.
If anybody has questions about how to AI-proof (at least in the short-term) an assignment ... I'm thinking of charging a premium for that, actually, as I keep hearing from my colleagues that nobody knows how to do it (further confirmation, I suppose, that I am a nobody!).
If you have questions about how ChatGPT helps you get away with using ChatGPT ... hell, leave a comment & I'll give you that for free!
