How to Tell if a Paper Was Written by AI?
Learn which writing patterns, citation issues, and other clues may suggest AI authorship. Understand the limits of AI detectors and why no single sign proves who wrote a paper.
Overview
There is no dependable visual clue or single AI detector that can prove who wrote a paper. Text detectors estimate whether a passage resembles text produced by language models; they do not observe the authoring process. A responsible assessment combines policy context, direct checks of claims and references, evidence of how the work developed, and a fair discussion with the writer. Detector scores and stylistic impressions can prompt questions, but should not be treated as verdicts.
“AI-written” can mean that a model generated whole passages, revised a human draft, translated or copy-edited it, or helped with research and planning. Those uses are not equivalent. Whether a particular use is allowed or must be disclosed depends on the applicable course, institution, journal or publisher policy.
What an AI detector can—and cannot—tell you
Most text detectors analyze the submitted text and produce a classification or score. They do not access the writer’s prompts, drafts or intent. Published research groups detection approaches into three broad types: trained classifiers, zero-shot methods that use statistical or language-model properties, and watermark detectors that look for a signal deliberately embedded during generation. A watermark is only available when the generating system inserted one, and edits can weaken it. (See the A Practical Examination of AI-Generated Text Detectors for Large Language Models - ACL Anthology and the GLTR: Statistical Detection and Visualization of Generated Text.)
| Approach | What it examines | Main limitation |
|---|---|---|
| Trained classifier | Patterns learned from examples labeled human- or machine-written | Results may not transfer to different models, subjects, languages or writing styles |
| Zero-shot statistical method | Properties such as token likelihoods or a model’s response to text perturbations | Human and generated writing can share these properties; results depend on text and model conditions |
| Watermark detector | A statistical signal placed in generated text by a participating model | The model must embed a detectable signal; text without one cannot be identified this way |
A detector’s output is not the probability that a named person used AI. It is a model-dependent classification. For example, Turnitin describes its displayed percentage as the share of qualifying prose that its model identifies as likely AI-generated or AI-altered; the percentage is separate from its similarity score. Its current guidance warns that results can misidentify human, AI-generated and AI-paraphrased text, and says the result must not be the sole basis for adverse action. Turnitin also suppresses numerical scores between 1% and 19% in its current report to reduce misinterpretation of a range where it says false positives are more frequent. (Using the AI Writing Report – Turnitin Guides.)
What published evaluations show
Detector performance is not one fixed property. It changes with the detector and version, underlying language model, length and genre of text, language, editing, and benchmark. A high score on a test set does not guarantee the same result for a different paper. False positives can harm a writer by attributing human writing to AI; false negatives miss generated writing. Both errors matter.
OpenAI’s 2023 evaluation of its own classifier illustrates this trade-off. On its English “challenge set,” it marked 26% of AI-written text as “likely AI-written” and incorrectly flagged human-written text 9% of the time. It said the classifier was very unreliable below 1,000 characters, could be confidently wrong, and could be evaded by editing. OpenAI discontinued the classifier on 20 July 2023, citing its low accuracy. These are results for that classifier and test set, not a universal estimate for every product. (New AI classifier for indicating AI-written text | OpenAI.)
A peer-reviewed 2023 study by Weber-Wulff and colleagues tested 12 publicly available detectors and two commercial systems using human writing, generated writing, machine translation, manual editing and paraphrasing. Its authors reported both false positives and false negatives, found machine translation could raise false-positive risk and paraphrasing reduced detection performance, and concluded that the tools they tested should not be used as evidence of academic misconduct on their own. The study focused on English-language material and described limitations in tool consistency and earlier research; its results are a particular evaluation, not a timeless ranking. (Testing of detection tools for AI-generated text | International Journal for Educational Integrity | Springer Nature Link.)
Newer methods do not remove the transfer problem. A 2025 NAACL Findings paper evaluated seven detector approaches on models and domains not used in their earlier evaluations. It found that a detector could have a seemingly strong overall AUROC yet perform poorly when required to keep false positives low; in some conditions, the true-positive rate at a 1% false-positive rate was 0%. This benchmark does not evaluate every commercial product, but demonstrates why a single advertised accuracy figure is insufficient. (A Practical Examination of AI-Generated Text Detectors for Large Language Models - ACL Anthology.)
A measured example of unequal false-positive risk
In a 2023 study, Liang and colleagues tested seven detectors on 91 TOEFL essays written by non-native English writers and 88 essays by US eighth-graders. The average false-positive rate across the seven tools was 61.22% for the TOEFL essays and 5.19% for the US student essays. The researchers linked the difference to writing patterns such as lower lexical variation, not to proof that any individual essay was generated. These rates apply only to those samples, detectors and study conditions; they are not general population error rates. (GPT detectors are biased against non-native English writers; AI-Detectors Biased Against Non-Native English Writers | Stanford HAI.)
Human readers are not reliable detectors either. In a controlled study of scientific abstracts, reviewers correctly identified 68% of generated abstracts but also incorrectly judged 14% of original abstracts to be generated. A separate study of gynecologic oncology abstracts found reviewers correctly classified 50% of the examples overall. Both studies used specific medical abstract sets, so their percentages should not be generalized to all readers or papers. (Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers; Association of reviewer experience with discriminating human-written versus ChatGPT-written abstracts.)
How to check a paper for AI: a practical process
A sound review starts with the paper’s claims and how it was produced, not with a search for an unmistakable “AI style.” If the question concerns misconduct, first determine which rules applied when the paper was submitted. If it concerns research reliability, verify facts and sources whether or not AI was used.
1. Define the question and applicable rule
“Was AI involved?” and “Was an academic rule broken?” are different questions. A paper may have been translated, grammar-checked or edited with AI without having its argument generated by AI. Conversely, generated passages may have been revised. Establish what use was prohibited or required to be disclosed, whether the policy was communicated in advance, and what evidence the relevant process accepts.
For research papers, authorship and disclosure policies also vary. The Committee on Publication Ethics says authors remain responsible for manuscript content and should disclose AI tools used in writing, image production, or data collection and analysis. Springer Nature’s current policy distinguishes uses it supports from opaque or non-verifiable uses and says responsibility cannot be delegated to AI. These are publisher policies, not universal rules for every journal. (Authorship and AI tools | COPE: Committee on Publication Ethics; Artificial Intelligence (AI) | Nature.)
2. Verify the paper’s sources and factual claims
Check references against the publisher, journal, DOI registry or library record. Confirm that each source exists, that its title and authors match the citation, and that it actually supports the sentence citing it. Compare quotations with the original text and edition. For empirical papers, check whether reported data, methods, tables and conclusions match the cited study or underlying dataset.
Fabricated or misdescribed citations, unsupported claims and incorrect quotations can occur in AI-generated work, but they can also result from human error, rushed research or misunderstanding. Their primary significance is that they reduce confidence in the paper’s scholarship; by themselves, they do not establish how it was written. University of Minnesota guidance identifies fabricated sources, quotations and data as possible issues to investigate while cautioning that textual features alone generally do not establish AI misuse. (Identifying and Responding to Unpermitted GenAI Use | Writing Across the Curriculum.)
3. Treat style differences as questions, not conclusions
Possible prompts for further review include a sudden change in voice within the paper, unusually generic treatment of a course-specific concept, claims that are broad but weakly supported, or a marked difference from the writer’s earlier work. None is a reliable standalone marker. Writers change style across genres and topics; editing, translation, collaboration, accessibility tools, stress and learning can all affect prose. A reader unfamiliar with a writer’s language background or normal range of expression may mistake unfamiliarity for evidence.
Do not infer authorship from a formulaic transition, polished grammar, repeated phrasing, em dashes, consistent paragraph length or a “too fluent” tone. Such features occur in human writing and are easy to alter in generated writing. The documented false-positive risk for non-native English essays shows why linguistic simplicity or constrained vocabulary must not be treated as proof of AI use.
4. Examine the work’s development, where appropriate
With permission and under applicable privacy rules, review dated outlines, research notes, drafts, tracked changes, document version history, source annotations and data-analysis files. A coherent progression can help establish how a paper developed. Large unexplained changes, source material that cannot be traced, or a mismatch between notes and final claims may warrant questions, but absence of drafts is not itself proof: a writer may work in another editor, draft offline or have no saved history.
Ask the writer to explain the argument, the choice of sources and how particular conclusions followed from the evidence. A fair discussion concerns the process and specific text, not a demand to reproduce the paper from memory. The University of Minnesota guidance notes that students may be more able to express ideas in writing than orally, especially multilingual or neurodiverse students; difficulty explaining something spontaneously should therefore be interpreted in context. (Identifying and Responding to Unpermitted GenAI Use | Writing Across the Curriculum.)
5. If using a detector, check what its result means
Use a detector only if the relevant institution permits it and there is a legitimate reason to do so. Before interpreting a score, check the product’s current documentation: supported languages, minimum text length, eligible content types, model/version, threshold and what the percentage denotes. Preserve the text and report version used if the result is part of a documented review, since tools can change.
A second detector’s agreement does not automatically create independent confirmation. Products may rely on similar patterns or training material, and their outputs are not necessarily statistically independent. Repeating a test with slightly different text or settings can produce a different result. Treat an output as a lead to review specific passages, never as the answer to who wrote the paper.
Why common “AI signs” are weak evidence
AI-generated writing can sound fluent and organized because language models produce likely continuations in context. But fluency, predictability and conventional academic phrasing are not unique to AI. Detectors that rely on these textual properties cannot directly identify an author or establish intent. Conversely, a human-edited AI draft may no longer display the patterns a detector expects. The 2023 multi-tool study found that paraphrasing reduced detection performance; the 2025 NAACL evaluation also found robustness varied across models and domains. (Testing of detection tools for AI-generated text | International Journal for Educational Integrity | Springer Nature Link; A Practical Examination of AI-Generated Text Detectors for Large Language Models - ACL Anthology.)
A common misconception is that an AI detector works like plagiarism software by finding a matching source. Similarity tools compare submitted text with indexed material; a generated-text detector instead estimates whether features of a passage resemble its model of generated language. The outputs address different questions. A low similarity score says little about whether AI helped write a text, and an AI score does not show that a passage was copied from a source.
Another misconception is that asking a chatbot whether it wrote the paper settles the question. Language models do not have reliable access to a complete record of a paper’s origin and can produce confident but unsupported answers. A detector’s label, a chatbot’s self-assessment and a stylistic impression are all weaker than verifiable provenance and corroborating process evidence.
Fairness, privacy and policy
A false accusation can affect a student’s grade, academic record or a researcher’s reputation. False negatives also matter, but they do not justify treating uncertain scores as proof. Assessors should apply the same written rules to all writers, consider language background and permitted support, document the evidence for a concern, give the author an opportunity to respond, and use institutional review and appeal procedures.
Uploading an unpublished paper or student submission to a public detector can expose personal information, confidential research, copyrighted work or intellectual property to an outside service. Check institutional approval, consent requirements, retention and reuse terms before uploading a document. The University of Queensland specifically advises staff not to upload student work to external AI detection tools because of privacy risks, and says AI must not act as the decision-maker. (UQ’s rules for using AI in assessment - Institute for Teaching and Learning Innovation - University of Queensland.)
Rules should define permitted assistance and disclosure expectations clearly. The University of Queensland’s assessment guidance, for example, distinguishes secure and open assessments and says instructions should state what tools are allowed and whether their use must be acknowledged. Use of AI is not automatically misconduct; the governing policy and the way the tool was used matter.
Conclusion
The most defensible answer to “how to tell if a paper was written by AI” is that authorship usually cannot be established from the final text alone. A detector can flag patterns for scrutiny but can misclassify both human and generated writing. Check the paper’s factual and bibliographic integrity, review process evidence when available, consider alternative explanations, and discuss the work fairly under the applicable policy. Treat an AI score as one limited signal—not proof of authorship or misconduct.