AI Detectors Still Get It Wrong. Here's How to Read the Score

AI writing detectors have become ordinary office software. Teachers run essays through them, editors check freelance copy, and hiring managers paste in cover letters. Most of the people using them never see more than a single number, usually a percentage, and a lot rides on how that number gets read. It is worth understanding what the tools actually measure, and where they reliably go wrong.

What a detector is measuring

A detector can't see who wrote a text. It looks at the text itself and asks whether it has the statistical habits of language-model output. Models tend to pick the most likely next word, so their writing is unusually predictable. They also write with an even rhythm, sentence after sentence of similar length, and they lean on a familiar stock of filler phrases that anyone who has read much chatbot output will recognize. A detector weighs signals like these and turns them into a score.

That approach works reasonably well on raw, unedited chatbot output of a decent length. Everywhere else? Much less well. And "everywhere else" is where most real-world cases live.

Where the scores go wrong

Formal and second-language writing. Careful human writing shares many of the same habits as machine text: even sentences, formal vocabulary, few contractions. Writers working in a second language often write this way on purpose; it's how they were taught. Researchers at Stanford found in 2023 that several popular detectors flagged more than half of a set of essays by non-native English speakers as AI-generated. Nobody had used a chatbot.

Edited text. A light paraphrase, or a person reworking a chatbot draft, removes most of the surface patterns. A low score means the text reads as human. It doesn't prove a human wrote it.

Short text. Below a few hundred words there is not much to measure, and scores swing wildly. A product description or a short email (a few dozen words) is close to unreadable for these tools.

Newer models. Each new generation of chatbot writes a little differently, and detectors trained on older output take time to catch up.

Be wary of the accuracy claim

Many detectors advertise accuracy of 98 or 99 percent. Those figures are usually measured on the vendor's own test set, often unedited model output against clearly human text. But that says little about a lightly edited essay by a student writing in their second language, which is exactly the case where a mistake does the most harm. OpenAI shut down its own AI text classifier in 2023 because its accuracy was too low, which tells you how hard the underlying problem is.

How to read a score sensibly

Treat the number as a signal about the text, never as a finding about a person. A few habits help a lot.

Look for tools that show their working. A score that comes with the specific sentences behind it can be checked by a human. A bare percentage can't. Pay attention to the middle of the scale, too. A text scoring around the midpoint is genuinely uncertain, and a tool that rounds that up to "likely AI" is overstating what it knows.

Get a second opinion from a tool that works differently. When two detectors built on different methods mark the same passages, that agreement means much more than either score alone. When they disagree, the evidence is weak. Simple as that.

Finally, weigh everything else you know. Draft history, a writer's earlier work, and a conversation about how the piece was written are all better evidence than any detector.

Some tools are being built around these limits rather than around a headline accuracy number. ScanForAI, a free checker, publishes the cases where its own scores are least reliable, marks the lines that pushed a score up, and declines to score languages it can't read properly rather than guessing. It's a sign that part of the industry is starting to describe these tools as what they are: useful instruments for raising a question, and poor ones for answering it.

AI detection is not going away, and used carefully it has real value. The risk is treating a probability as a verdict. Anyone acting on these scores should know the difference.