AI Tools

How AI Content Detectors Work: Burstiness and Perplexity Explained

AI content detectors have become a popular topic — teachers use them to check student submissions, editors use them to verify article authenticity, and marketers use them to review outsourced copy. But most people using these tools have no idea how they actually work.

This article explains the two main signals that most AI detectors measure: perplexity and burstiness. Once you understand what these mean, you'll also understand why no detector is perfectly accurate — and what that means for how you should interpret the results.

What AI Language Models Actually Do

To understand how detection works, it helps to first understand how AI models generate text. When you ask ChatGPT or Claude to write something, the model is essentially predicting the next word based on everything that came before it — over and over, one token at a time.

These models are trained on enormous amounts of text, so they've learned which words and phrases tend to follow other words and phrases. When they generate text, they tend to pick statistically likely, "safe" word choices. This produces text that's fluent and grammatically correct — but also somewhat predictable.

Human writers, on the other hand, make idiosyncratic choices. They use unusual metaphors, switch sentence lengths unexpectedly, make small grammatical quirks, and bring in references that surprise you. This unpredictability is what detectors try to measure.

Perplexity: How Predictable Is the Text?

Perplexity is a measure of how surprised a language model is by a sequence of text. Low perplexity means the text was easy to predict — the kind of output an AI model would naturally generate. High perplexity means the text was harder to predict, suggesting a more idiosyncratic human writer.

AI detectors run your text through a language model and measure how easily the model could have predicted each word. If the text scores consistently low perplexity, it's flagged as potentially AI-generated.

Burstiness: How Varied Is the Sentence Structure?

Burstiness refers to the variation in sentence lengths throughout a piece of writing. Human writing tends to be "bursty" — you might have two very short sentences, then a long complex one, then a medium one. This irregular rhythm is natural to how people actually think and write.

AI models tend to produce text with much more uniform sentence lengths. They're optimized for readability, which means they gravitate toward sentences in the 15–20 word range. The rhythm is smooth — but also monotonous.

Mathematically, burstiness is measured using the standard deviation of sentence lengths. A high standard deviation means sentence lengths vary a lot (more human-like). A low standard deviation means sentences are uniformly similar in length (more AI-like).

Why These Signals Work (But Not Perfectly)

These two metrics — perplexity and burstiness — capture something real about the difference between AI and human writing. But there are important limitations:

• Some humans write uniformly. Technical writers, people writing in a second language, or anyone following a strict style guide may produce low-burstiness text that detectors flag as AI.
• AI can be prompted to write with variation. If you specifically ask an AI to use varied sentence lengths and unconventional word choices, it can do so — reducing its detectability significantly.
• Short text is unreliable. With fewer than 100–150 words, there isn't enough data to calculate meaningful statistics. Most detectors perform poorly on short passages.
• Edited AI text is harder to detect. If a human edits and rewrites AI-generated text, the statistical fingerprints get blurred.

What Other Signals Do Detectors Use?

Beyond perplexity and burstiness, some detectors look at additional patterns:

• Transitional word density: AI models frequently use words like "furthermore," "moreover," "it's worth noting," and "in conclusion." High density of these phrases is a soft signal for AI authorship.
• Vocabulary diversity: AI text sometimes uses a narrower range of vocabulary than a human expert would, especially on specialized topics.
• Structural predictability: AI-generated articles often follow very predictable structures — introduction, numbered points, conclusion — that can be identified as a pattern.

How to Interpret Detection Results

If you're using an AI detector, treat the result as a signal, not a verdict. A 70% AI probability score doesn't mean the text is definitely AI-generated — it means the text has statistical characteristics that overlap with AI output.

The most reliable way to use these tools is to look at them in context. A piece of text that scores high on AI probability and was written suspiciously quickly by someone with no track record of that writing quality is worth a closer look. A piece that scores moderately high from a known skilled writer may simply be their natural efficient style.

Frequently Asked Questions