Pangram is reliable on long English prose according to the evidence we found, but it is not infallible, and the evidence is thin. Pangram’s own documentation reports very low error rates, and one independent working paper found it performed best of the four detectors it compared. Pangram also says it is less dependable on lists, templates, reference sections and short text. Treat a result as a pointer to passages worth reading, not as proof of who wrote them.
Disclosure: AI-Detection.io runs on the Pangram Labs API, so we are a customer of the product described here. We have not run our own accuracy tests, and this page is not an independent review.
Key takeaways
- The low error rates come mostly from Pangram’s own technical report and model cards. They describe specific datasets and model versions.
- One independent working paper, by University of Chicago researchers, found Pangram had essentially no false positives at most thresholds and false negatives of 2 to 4% on the writing it sampled.
- Pangram’s own model card lists where false positives are more likely, and says the error rate is not zero.
- “Not reliable” claims about Pangram usually come down to the cases above: short, templated or heavily edited text.
What does “reliable” mean for a detector?
Two separate errors matter. A false positive labels a person’s writing as AI. A false negative misses AI text. A tool can be strong on one and weak on the other, so a single accuracy figure hides the trade-off. For a student or a writer, the false positive is the costly error. How accurate AI detectors are explains why the answer also depends on length, genre and the model that produced the text.
What does Pangram report about its own accuracy?
Pangram’s technical report and its Pangram 3.2 model card report false positive rates between 0.00% and 0.54% across its human-text datasets, and false negative rates between 0.00% and 1.98%. These are the vendor’s measurements on the vendor’s datasets, for one model version. We do not know which version serves a given API request on a given day. The full breakdown, including the homepage’s different headline figure, is in what the Pangram AI detector is.
What do independent tests show?
The strongest independent evidence we found is the Jabarian and Imas working paper, Artificial Writing and Automated Detection, summarized in Chicago Booth Review. It compared Pangram, GPTZero, Originality.ai and an open-source model on about 2,000 human-written passages across six kinds of writing.
- Pangram had essentially no false positives at most thresholds and false negatives of 2 to 4%.
- GPTZero’s false positives stayed below 1%, with low false negatives.
- Originality.ai’s false negatives ran from 10% to 40% depending on the model.
- All three commercial tools lost accuracy on passages under 50 words.
It is one study, on the genres it sampled. It does not cover Copyleaks, other languages or later model versions. We read the abstract page and the Chicago Booth summary, not the full paper.
Where does Pangram say it is less reliable?
The Pangram 3.2 model card says the model is meant for long-form writing in complete sentences. It lists bullet lists, instructions and technical manuals, tables of contents, reference sections, templated writing and dense equations as more prone to false positives. It also warns that short human passages between AI text may be labeled AI-assisted, and notes that PDFs can introduce parser errors. These are the situations to doubt first.
Why do some people say Pangram is not reliable?
We cannot know every reason, and we have no independent data on individual complaints. The documented causes of a disputed result are the ones above: short samples, lists and templates, heavily edited or mixed text, and the soft line between “AI-assisted” and “AI-generated”. A low rate across a dataset still leaves some individual writers flagged, and that is why can AI detectors be wrong and AI detection false positives in student writing matter more than any headline figure.
How should you use a Pangram result?
- Check the length and genre first. Under a couple of hundred words, or in a list or template, treat the result as rough.
- Read the highlighted passages in context, and the confidence label on each.
- Look for other evidence: drafts, version history, sources and a conversation. What to do when an AI detector flags student writing sets out a fair process.
- Never use the number alone as the basis for a penalty.
To see a result for yourself, use our AI-generated text detector. For other tools, compare GPTZero, Copyleaks and Originality.ai, or read which AI detector teachers use.
FAQ
Is Pangram more reliable than GPTZero? On the one independent study we found, Pangram and GPTZero were close on false positives and Pangram caught more AI text. That is a single comparison on specific writing, so it does not settle the question for every kind of text or for later versions.
Can Pangram give a false positive on my own writing? Yes. Its error rate is not zero, and it is higher on the kinds of text Pangram lists above. A flag on your work is a reason to review the passages, not proof of AI use.
Can I trust Pangram’s 99% accuracy claim? Treat it as a marketing figure. Pangram’s pages quote different numbers, and the per-dataset tables in its model card say more than a headline.
Does this site publish its own accuracy figure for Pangram? No. We have not tested it ourselves, and the accuracy page explains why we will not borrow a number.