How accurate are AI detectors?

Nobody knows a single accuracy number for AI detectors, because there is no single accuracy. A classifier's score depends on which model wrote the text, the subject, the length, whether the text was edited, and who the human writers in the test set were. Every vendor publishes figures on its own test set with its own threshold, so the numbers are not comparable. The one peer-reviewed fairness study found an average 61.3 percent false positive rate on essays by non-native English speakers. The fairest comparison available is the RAID benchmark, where detectors are scored on the same data at the same false positive rate, and where results vary widely by attack.

What a detector actually does

Commercial detectors are classifiers: models trained on piles of human text and model text to tell the two apart by style, predictability and structure. They need no key and work on any model's output, which is their advantage over watermarks. Their weakness is the same as any classifier's: they learn the past. A new model, a new domain, a new rewriting tool, or a writer whose style resembles the training data's machine side will push the score the wrong way until the vendor retrains. Zero-shot methods such as Binoculars avoid training data by comparing how two language models react to the text, which makes them reproducible, but they face the same fundamental limit.

Two errors, one trade-off

A detector can flag human text as AI (a false positive) or miss machine text (a false negative). Moving the threshold trades one for the other. A vendor who advertises 99 percent detection is usually quoting the miss rate at a threshold that produces more false alarms than it mentions, and a vendor who advertises a 1 percent false positive rate is usually quoting the alarm rate at a threshold that misses more. The only honest way to compare two detectors is at the same false positive rate on the same data. RAID does that; marketing pages do not.

The false positive problem

In 2023 Liang and colleagues at Stanford ran seven detectors over TOEFL essays by non-native English speakers and essays by American eighth graders. The native essays passed. The TOEFL essays were flagged at an average rate of 61.3 percent, and more than 91 percent of them were flagged by at least one tool. The cause is perplexity: detectors read predictable word choice as machine-like, and writers working in a second language produce predictable text. Asking a model to enrich the essays' vocabulary flipped the results. Vendors have retrained since, and some now publish their own non-native figures, but none has published a comparable peer-reviewed test. The same mechanism affects writers with plain styles, writers using templates, and anyone who writes the way textbooks do.

Why vendor numbers disagree with each other

Every vendor evaluates on text it chose, generated by models it chose, at a threshold it chose, against human text it chose. Many then compare competitors on that same private set. Affiliate review sites repeat the results. The vendors most worth reading are the ones that publish datasets and method, which in this directory means Originality.ai's benchmark studies and Pangram's method posts, and the one worth trusting most is the shared leaderboard neither controls.

What accuracy looks like after editing

Independent tests consistently find that paraphrasing cuts classifier accuracy hard, sometimes to near chance. Vendors responded by training on humanizer output: Turnitin's bypasser detection since August 2025, Pangram's humanizer detection since early 2025. That has raised detection of rewritten text and will keep changing as rewriters retrain against detectors. A score from last year says nothing about this year, which is why this directory dates every entry.

How to use a score

Use more than one detector, and expect them to disagree. Treat a score as a reason to look at process evidence, never as the evidence. Read the vendor's own guide; Turnitin's says its percentage is an indicator that should start a conversation. Check the length: anything under a few hundred words is noise. And when the stakes are a person's record, follow the guide for people who have been flagged, whichever side of the table you sit on.

Questions people ask

What is a good false positive rate for an AI detector?

Vendors quote 1 percent or lower. Even 1 percent means one innocent student in a hundred on every assignment, and published tests on non-native writing found rates far higher.

Why do two detectors give different scores on the same text?

They are different classifiers trained on different data with different thresholds. Disagreement is normal and is itself a reason not to rely on one score.

Is there an independent AI detector benchmark?

RAID, from the University of Pennsylvania, scores detectors on more than six million generations with eleven attacks at a fixed false positive rate, on a public leaderboard.