A student turns in an essay she wrote herself over three late nights, and the professor's AI detector comes back with "87% likely AI-generated." A technical writer with fifteen years of experience gets the same verdict on a user manual he has written in the same clipped, consistent style since before large language models existed. Neither of them used AI. Both of them now have to prove a negative.
This happens more often than detector vendors like to admit, and it is not because the tools are broken. It is because of what they are actually measuring. An AI detector does not detect AI the way a metal detector detects metal. It measures statistical properties of text and estimates a probability. Some human writing styles just happen to share those properties with machine output. Understanding what is being measured is the only way to read a detector score sensibly instead of treating it as a verdict.
Photo by jessica olivella on Pexels
What a Detector Actually Measures
Most AI detectors, including the statistical models underlying commercial products, look at a handful of measurable properties in a chunk of text rather than reading it for meaning.
Perplexity is the first. It is a measure of how predictable each next word is, given the words before it, according to a language model. Text with very low perplexity uses common, expected word choices at every turn. Language models are trained to minimize exactly this kind of surprise, so their raw output tends to sit in a narrow, predictable band. A human writer who favors plain, common words and short, conventional sentence structures produces text with similarly low perplexity, for entirely different reasons.
Burstiness is the second. Human writing tends to vary a lot between sentences: a long, winding sentence followed by a three-word fragment, then a medium one. Machine output, especially from older or less-tuned models, tends to produce sentences of more uniform length and rhythm. Burstiness measures that variance. Low burstiness reads as "AI-like" even when a careful technical editor produced it on purpose, because technical writing style guides often ask for consistent, parallel sentence structure.
Vocabulary diversity is the third. This tracks how repetitive the word choices are across a passage. A writer working from a fixed template, a strict style guide, or a narrow technical vocabulary will naturally reuse the same terms, which lowers diversity scores in a way that overlaps with machine-generated text.
The Stanford Human-Centered AI Institute has published research questioning the reliability of these statistical signals precisely because none of the three properties is unique to machine text. They are correlated with it, not caused by it.
Who Gets Falsely Flagged Most Often
The false positive rate is not evenly distributed. Some kinds of human writers hit these statistical patterns more than others, through no fault of their own.
Non-native English speakers are flagged disproportionately. Research out of Stanford in 2023 found detectors misclassified a meaningful share of essays written by non-native English speakers as AI-generated, because those writers often use simpler vocabulary and more conventional sentence patterns, the same features that lower perplexity and burstiness scores. This is one of the most consequential false-positive patterns because it disproportionately affects students and job applicants who are already navigating a language barrier. The Educational Testing Service has published guidance for institutions on why a single automated score should never be the sole basis for an academic integrity decision, precisely because of this kind of skew.
Technical and procedural writers are flagged often too. A style guide that mandates active voice, consistent terminology, and short declarative sentences produces exactly the low-burstiness, low-perplexity profile a detector associates with machine text. The writer is following instructions, not hiding anything.
People who write in a second draft after heavy self-editing also trend toward flags. A first draft is naturally bursty and uneven. A writer who tightens every sentence for clarity, cuts filler words, and standardizes phrasing across a document is, unintentionally, moving the text's statistical fingerprint closer to what a language model produces, because both processes optimize toward the same kind of clean, predictable prose.
The mistake most people make with a detector score is treating it like a lie-detector result instead of a probability estimate built on three fuzzy signals. Read it as one data point, not a verdict. - Dennis Traina, founder of 137Foundry
Why Detectors Still Have a Real Job to Do
None of this means detectors are useless. On text that is heavily AI-generated with no editing at all, the statistical signature is often strong and consistent, and a detector will correctly flag it at a high rate. The failure mode is specifically in the middle ground: lightly edited AI text, or human text that happens to share the machine's statistical habits.
Editors and teachers who use detectors well treat a high score as a prompt to look closer, not as a conclusion. They check for other signals a statistical model cannot see: does the writing show a specific, verifiable personal experience? Does it cite something the model could not know? Does the structure match how this specific person has written in the past? A detector score paired with a plagiarism-style comparison against a person's known writing history is far more reliable than either signal alone.
Photo by Alex Dos Santos on Pexels
How to Read Your Own Detector Score
If you run your own writing through a detector and get flagged, a few checks help you figure out whether the flag is a real signal or statistical noise.
Look at your sentence length variance across the piece. If every sentence is roughly the same length, that alone will drag your burstiness score down regardless of who wrote it. Deliberately varying sentence length in a revision pass, mixing short punchy sentences with longer explanatory ones, changes the statistical profile without changing the substance.
Check whether you were following a rigid template or style guide. Detector scores on templated writing, like standardized report formats or documentation with mandated phrasing, are close to meaningless because the template itself creates the low-diversity signature. This is worth explaining upfront to anyone relying on the score, rather than discovering it after the fact.
Consider the editing history. If you have drafts, revision logs, or version history showing the piece evolving over multiple sessions, that is stronger evidence of authorship than any detector score in either direction. Detectors look at a single snapshot of text; they cannot see the process that produced it.
Run a second detector if the stakes are high. Different tools weight perplexity, burstiness, and vocabulary diversity differently, and a piece that trips one tool's threshold may sit comfortably under another's. Disagreement between tools is itself useful information: it tells you the signal is borderline, not conclusive.
What This Means for Institutions Using These Tools
Schools and employers who adopt AI detectors as a hard gate, an automatic fail or automatic rejection based on a score, are building policy on a tool that was never validated for that use. The OpenAI's own retired AI text classifier writeup acknowledged low reliability on short texts and non-English writing before the company discontinued the tool for exactly this reason. Most detector vendors have quietly softened their marketing language from "detects AI" to "estimates likelihood" over the past two years, which is a tell about how the internal confidence has shifted.
A defensible policy uses a detector score as one input that triggers a human conversation, not an automatic penalty. Pair it with a request for drafts, an in-person discussion, or a comparison against the person's prior writing samples. None of that is as fast as an automated score, but automated speed is exactly what produces false accusations against careful writers and non-native speakers.
Photo by AI25.Studio Studio on Pexels
Using a Detector on Your Own Work Before You Publish
If you write for a living and want to know your own baseline, running a draft through a detector before submitting it is a reasonable habit, not paranoia. The AI Content Detector on this site scores pasted text against the same perplexity, burstiness, and vocabulary signals covered above and shows you which sentences are pulling the score in which direction, so you can see whether a flag is coming from your actual style or from a specific over-templated section.
That kind of transparency, seeing the sentence-level breakdown rather than a single opaque number, is the difference between a tool you can act on and one that just produces anxiety. If a paragraph is flagged because it is a boilerplate disclaimer you copied verbatim, you now know to leave it alone. If it is flagged because you wrote five sentences in a row of identical length, that is an easy, useful edit.
The Short Version
AI detectors measure perplexity, burstiness, and vocabulary diversity, three statistical properties that correlate with machine-generated text but are not exclusive to it. Non-native English speakers, technical writers following strict style guides, and heavily self-edited human prose all tend to share those same statistical patterns, which produces real false positives on real human writing.
Treat a detector score as a prompt to look closer, never as a verdict on its own. Check sentence variance, consider whether a template shaped the text, and look at editing history before accepting or rejecting a score. For an accessible way to check your own writing's statistical fingerprint before you publish or submit it, try the AI Content Detector, or browse the full EvvyTools tools directory for related writing tools. For more breakdowns like this one, the EvvyTools blog covers the tools behind the numbers.
