AI Series Part IV
By Bob Deakin
May I have a word with you after class, Mr. Deakin? A student may suspect they are in trouble if they receive such a request after handing in a written project. If it’s their writing, nothing to worry about. If they wrote it with an AI generator, they may have to explain in their own words. So, how do teachers know when students are using AI?
There Is No Such Thing As a Lie Detector
As educators know, no detection tool is guaranteed, and AI generators are improving daily. Increasingly, institutions and school systems are offering guides to deal with AI generation and detection, such as Cornell University’s Generative Artificial Intelligence for Education and Pedagogy.
Ten AI detection tools currently used by educators:
ChatGPT Content Detector (free)
Compilatio (free)
ContentAtScale (free up to 25k characters)
Copyleaks (free)
Crossplag (subscription)
GPTZero (free up to 5k characters)
Passed.AI (subscription)
PlagiarismCheck (free)
Winston AI Detector (subscription)
Writeful (free)
Each tool has its pluses and minuses, but mere familiarity with one can help an educator (or student) develop their indicators for AI. Looking for more examples? Montclair University offers a short list of Red Flags.
Harvard Dean of Science Christopher W. Stubbs voiced his take in a September Harvard Crimson article: “I don’t think there is a one-size-fits-all course policy here,” Stubbs said. “What we’re asking the faculty is that they become informed, that they understand the impact this has on the learning objectives for their courses, and then importantly, that they communicate to students clearly and often what their course policy is.”
At The Same Time…
Educators know what to look for, so what if the normally wayward student turns in content worthy of publication? Then it’s time to test the teacher’s sleuthing skills.
At the same time, the level of inexperience a student displays in a document may mimic the robotic tendencies of AI, providing a conundrum. Students translating words to English from a different language may also produce content that appears to be AI-generated.
Using AI to detect AI - possibly from the same source that generates it - is an unintended ambiguity. There are enough lessons to learn without adding irony.
How Students Beat the System
How do you bypass detection with your AI-generated content? One method is to use a detection bypass tool such as Quillbot, Stealthwriter, or Undetectable AI.
Each provides a broader base of AI-generated content based on input. You enter content - human or AI-generated - then determine what level of sophistication you desire. It takes no more time than AI generators, and you can repeatedly enter the text to achieve the result you are striving for.
Another bypass technique is to give your AI generator more thorough instructions. This can be as simple as “Describe a trip to the Golden Gate Bridge in the style of Hunter S. Thompson.”
Here’s what CopyAI produced from that prompt:
The sun was dipping below the San Francisco skyline as I embarked on a tumultuous odyssey towards the legendary Golden Gate Bridge. I had heard rumors of its ethereal allure, beckoning the wanderers and misfits like a siren in the night. With my mind clouded by a potent cocktail of adrenaline and curiosity, I surrendered to the call of this mythical structure.
Just Be Nice
Like anything else, using AI generation is a skill to develop. Believe it or not, the nicer you are, the more likely it will produce for you. I submitted the following prompt to Rytr: “Please describe smartphone manners in the style of Oprah Winfrey.”
Here’s what I received:
Smartphone Manners: Embracing the Oprah Winfrey Way
Oh, my darlings! Let's talk about smartphone manners, shall we? In this digital age, where our trusty smartphones have become extensions of ourselves, it's essential that we navigate this technological landscape with grace and consideration. So, let me share with you the Oprah Winfrey way of embracing smartphone etiquette.
Imagine the possibilities! The key is to understand how the algorithms work.
Teachers Don’t Have Time to Be Detectives
AI detection will organically improve following the advances in AI generation, but the generators are winning the race. However, teachers don’t become teachers to become detectives.
“I don’t want to be in an adversarial relationship with my students,” said Gina Parnaby of the Marist School near Atlanta in a New York Times story. “If our mind-set approaching this is that we have to build a better mousetrap to catch kids cheating, I just think that’s the wrong approach, because the kids are going to figure something out.”
What Is the Endgame with Generative AI in Schools?
There is no need to write code to produce human-sounding copy, just patience and imagination. This is what separates the expert imposters from the lazy ones.
You might wonder, “If someone is that creative with deception, wouldn’t they make a good writer?”
Perhaps, but that involves research, construction, style, spelling and grammar checks, editing, economizing, and reviewing to get it right. By then, the AI generation expert is already pondering the release date of their next research paper, novel, script, or YouTube video.
It ain’t whatcha write, it’s the way atcha write it - Jack Kerouac
In conclusion, humans are compelled to unmask deception when we see it. If it is presented as AI, no problem. However, if it is supposed to be from the heart, May I have a word with you after class, Mr. Deakin?
###
Three primary reasons why AI Detectors fail:
Lack of training data: For good classifiers, AIs require extensive training data to analyze text accurately. Insufficient or biased datasets can lead to poor detection performance.
Language model limitations: AI-generated content has become increasingly sophisticated, making it difficult for detectors to distinguish between human-written and machine-generated text. As AI writing tools like GPT-3 continue to improve, the challenge of detecting AI-written text will only grow more complex.
Evolving techniques: As AI writers learn from their mistakes and adapt new strategies, they may outpace the ability of detection algorithms – requiring constant updates and refinements in response.
Courtesy Julia McCoy of Content Hacker