With more than 100 million active users, ChatGPT has sparked a true revolution in artificial intelligence. We have previously explained how to connect this chatbot to the web and how to maintain a more or less fluid conversation... but there are also many conflicts. One of them is the use of ChatGPT to write essays, solve problems, and prepare for school exams. Traditional education feels under threat; however, its “high-tech solution” is nothing other than using artificial intelligences to detect generated content. Today we will explore two free options.
ChatGPT: A “Rincón del Vago 2.0”?
The story of the Rincón del Vago is truly fabulous. The original project was created by two students tired of fighting against obsolete programs, and in a short time the portal achieved worldwide fame, in addition to being involved in several high-profile scandals. Today, the feeling among users is that a “Rincón del Vago 2.0” has emerged... and it is called ChatGPT. While no one denies that the chatbot commits a huge number of errors (and some very rude ones), the latest experiments indicate that it is good enough to pass exams in different subjects... and teachers are not happy.
Solutions? The most popular opinion is to use an artificial intelligence to recognize another. Of course, that is not free from challenges, and an overly strict algorithm could generate a wave of false positives. However, we already have two detectors at our disposal. The first is AI Text Classifier, developed by OpenAI, the same creator of ChatGPT. And the second is GPTZero, designed by Edward Tian from Princeton University. Both are free, and the only thing they ask for is logging in with an account (or alternative credentials like Google).
AI Content Detectors: How Well Do They Work?
My “benchmark” is based on five topics: Superdeterminism, the element uranium, Caesar salad, the French service Minitel, and the Volksempfänger radio from Nazi Germany. After asking ChatGPT for a description of each one, I copied and pasted the text of each (you can see it here) into AI Text Classifier and GPTZero.
AI Text Classifier
Immediately we encounter one of the biggest problems these detectors will have to face (and solve): Their conclusions are limited. AI Text Classifier said that its analysis of the superdeterminism text is “unclear”, meaning it could not clearly determine if it was written by an artificial intelligence. For the uranium text it used the expression “likely”, but with Caesar salad we discovered another restriction: The text must be 1,000 characters or more. Finally, the Minitel and Volksempfänger texts were rated as “possibly”. The five descriptions from AI Text Classifier are “very unlikely”, “unlikely”, “unclear”, “possibly”, and “likely”.
GPTZero
GPTZero is a bit more transparent with its analysis. At the bottom of the interface it explains that it uses two indices: one of perplexity, and the other of “explosion” or burstiness. The perplexity of a document is the measure of randomness in the text, while the “explosion” measures the variation of perplexity. With a combination of these indices, GPTZero gave four texts a rating of “likely”, and for the uranium text it used the expression “may include parts”. Regarding its restrictions, GPTZero takes a more relaxed position, and asks the user for a minimum of 250 characters in the text.
In Summary
Let's start with the obvious: The genie is out of the bottle. Millions of students around the world will try to use ChatGPT and other models to reduce their obligations. Some teachers have already responded with a “return” to oral exams and handwritten assignments, but that is an unviable strategy in the long run. Furthermore, the information can be adapted, and nothing prevents a student with a good knowledge of English from getting a summary from ChatGPT, and then presenting their own translation that will be virtually undetectable.
Right now, both AI Text Classifier and GPTZero are pointing in the right direction, however, they are not in a position to offer guarantees to teachers, quite the opposite. Detectors will gain capability and accuracy, but so will chatbots. If you allow me the hyperbole, “the cat and the mouse” are preparing to throw algorithms at each other. To be continued...
AI Text Classifier: Click here
GPTZero: Click here
