In theory, AI detectors analyze a piece of writing and assess what percentage of the text is AI-generated versus human-generated. However, multiple studies have shown that AI detectors were "neither accurate nor reliable," producing a high number of both false positives and false negatives.
False positives incorrectly flag content written by humans as having been written by a generative AI tool. False positives and accusations of academic misconduct can have serious repercussions for a student’s academic record. False positives can also create an environment of distrust where students are treated as suspicious by default and that can undermine the faculty-student relationship.
False positive rates vary widely. Turnitin has previously stated that its AI checker had a less than 1% false positive rate though a later study by the Washington Post produced a much higher rate of 50% (albeit with a much smaller sample size). Recent studies also indicate that neurodivergent students (autism, ADHD, dyslexia, etc…) and students for whom English is a second language are flagged by AI detection tools at higher rates than native English speakers due to reliance on repeated phrases, terms, and words.
False negatives fail to identify documents that do contain AI-generated text. False negatives occur most often due to an AI tool’s sensitivity settings or to users intentionally using evasive techniques to make their text more human-like.
First, AI detection companies need to balance the false positive and false negative rates in light of the serious academic ramifications that result from false positives. For example, Turnitin’s AI checker can miss roughly 15 percent of AI-generated text in a document. “We’re comfortable with that [false negative rate] since we do not want to highlight human-written text as AI text,” says the company, noting its 1 percent false positive rate.
Second, individuals are able to circumvent AI detection tools by simply paraphrasing, inserting emotion or anecdotes, increasing word or structure diversity, or simply using other AI tools (e.g. Writesonic’s AI Humanizer or UndetectableAI) to add human-like elements to their writing. Cat Casey, chief growth officer at Reveal and a member of the New York State Bar AI Task Force noted,"I could pass any generative AI detector by simply engineering my prompts in such a way that it creates the fallibility or the lack of pattern in human language.” She added that she is often able to fool detectors 80-90% of the time simply by adding the single word “cheeky” to her prompt since it implies irreverent metaphors.
AI generators and AI detectors are locked in an eternal arms race, with both getting better over time. “As text-generating AI improves, so will the detectors — a never-ending back-and-forth similar to that between cybercriminals and security researchers… That’s all to say that there’s no silver bullet to solve the problems AI-generated text poses. Quite likely, there won’t ever be.” TechCrunch (January 31, 2023)