RAID benchmark
A shared benchmark of more than six million generations across models, domains, decoding settings and adversarial attacks, including paraphrasing. The public leaderboard is the closest thing to a neutral scoreboard for detectors. Paper at arXiv 2405.07940.
What it is
RAID is the benchmark that stops vendors grading their own homework. It contains more than six million generations across many models, domains, decoding strategies, repetition penalties and eleven adversarial attacks including paraphrasing, synonym swaps and homoglyphs. Detectors are scored on a public leaderboard at a fixed false positive rate, which is the only fair way to compare them. If a vendor's accuracy claim is not on a shared benchmark like this, discount it.
When to use this entry
Use it to compare detectors on equal terms and to see how each attack hurts them.
Questions
Where can I compare AI detectors fairly?
The RAID leaderboard scores detectors on the same data at the same false positive rate.
Which attacks does RAID include?
Eleven, including paraphrasing, synonym replacement, misspelling, whitespace, homoglyphs and zero-width characters.