Contextuality and Contradictions: Exploring Real-World Identifications of AI-Generated Writing
Organizations: University of Vermont, USA
Abstract
As AI-generated content spreads into nearly every domain of human interaction, practices and discourses of AI detection, both formal and informal, have become an everyday part of how people interact with text and with each other. In this paper, we explore the real-world practices and discourses of AI identification that have emerged in response to a shifting discursive landscape where the human provenance of a text can no longer be taken for granted. We pair qualitative coding with computational analysis of lexical change, grounded in linguistic anthropological perspectives on language and communication, to investigate AI identifications among English Wikipedia editors and students. We find that these identifications draw on a diversity of evidentiary types, from discrete textual features and judgments about content to circular sense-making claims and presumptions about the author's moral character, and that they are contradictory, dynamic, and contextually dependent. In practice, what often motivates an AI identification is not the presence of any specific textual element, but pragmatic infelicity: a perceived mismatch between the elements present and those deemed appropriate within the text's particular context. Consequently, real-world identifications of AI-generated writing do not merely neutrally point out the perceived presence of "AI-like" features, but can be leveraged as a discursive resource to maintain and define sociolinguistic norms: what can be said in a given context, what is appropriate, and, ultimately, what is human. Further, the design and implementation of systems and tools that treat AI-likeness as a context-free property of text may therefore inherit, and enforce at scale, the norms of whichever contexts they were trained on.
Figures & tables
| Unusual / Overwrought | Bland / Simplistic |
|---|---|
| “too many unnecessary fancy words and metaphors”; “too detailed”; “flowery language”; “uncommon words and grammar”; “Advanced language”; “Very long and descriptive”; “long sentences” | “the writing style is very basic”; “bland”; “plain”; “very plain”; “low sophistication in vocabulary”; “Short, punchy sentences”; “a lot of short sentences” |
| Errors Present | Errors Absent |
| “often has foolish and simple mistakes”; “errors in grammar or lots of weird grammar”; “weird grammar. Strange wordings”; “inaccurate”; “It can have major errors […]” | “it doesnt have any spelling/grammar mistakes”; “lack of grammatical and syntactical errors”; “smart”; “[if] their writing is too good people will get skeptical and say it’s ai” |
| Friendly / Polite | Formal / Academic |
| “polite”; “overly friendly”; “overly sweet and solicitious”; “overly nice”; “It feels like it is being weirdly friendly”; “a high amount of politeness” | “very acedemic language style”; “very formal style of writing”; “seem overly formal”; “uses grammar that is often believed to be extremely formal” |
| Students’ AI Indices | Wikipedia Editors’ AI Indices |
| “neutral tone, playing both sides”; “super neutral”; “lack of opinions”; “refusal to commit to an opinion”; “no emotion”; “doesn’t lean into emotions”; “Unemotional”; “it takes the emotions out”; “not really having an opinion” “lack of personal connection between text and author”; “doesn’t seem like a person who cares about the issue”; “soulless”; “sounds like the smile given by a depressed chick fil a employee as they tell you my pleasure, except the soul of that employee has been replaced by a rock.” | “non-neutral language”; “random adjectives that feel opinionated”; “editorializing tone”; “undue emphasis on significance”; “undue emphasis on importance”; “promotional tone”; “extreme promotional tone”; “Tone seems very promotional and unencyclopedic”; “significant puffery”; “peacock phrasing”; “fake profundity”; “too enthusiastic” |
| Evidentiary motivations | Social motivations |
| “not really having an opinion” “because of the m dashes and the knowledge that the average person wouldn’t have” “those dash things” “the em dashes and the sentence structure” “Because of the opening line ‘I get that it can feel…’ lots of LLMs kinda repeat things back when they start a response” “unnecessary use of adjectives” “A but B” | “to make [Person A’s] opinion known” “accuses Person B of communicating without emotion” “shuts down the ability to bring genuine concerns up since you will be deemed ‘fake/inhuman’ ” “minimizing”, “minimizes the comment” “shut [Person B] down using logical fallacy/attacking the speaker not their argument” “It dismisses the valid claims made by person b. Any criticism can be dismissed making learning/improving as a person harder” |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Group | Population | Timing | Total N | Min No. | Inc N | Min Inc No. |
|---|---|---|---|---|---|---|
| Adult (18+) | ||||||
| City HS students (2 classes) | Pre Module 2, October 26 | 4 | 4 | 4 | 4 | |
| Undergraduates | Post Day 3, July 10 | 12 | - | 12 | - | |
| Librarians | Pre, June 19–22 | 5 | 5 | 0 | 0 | |
| Librarians | Post, July 7 | 3 | - | 0 | - | |
| Undergraduates | Pre Day 1, June 11 | 15 | 15 | 15 | 15 | |
| Category | Sub-type | Description | Example |
| Text. Features | Lexicon | Word choice | “used highlight as a verb”; “flowery” |
| Orthography | Writing convention | “excessive dashes” | |
| Structure | Grammar, syntax, or text formatting | “markdown dividers” | |
| Quality | Descriptive or evaluative statement about the textual features as a whole | “high density of AI vocabulary words” | |
| Error | Mistakes in grammar or usage | “grammatical errors” | |
| Circular | Refers to AI and/or presumes the existence of a knowable register | “prose reminiscent of an essay/thesis written by a language model” |
| Sub-band ( ) | Labels |
|---|---|
| Lexicon (61) | “accentuates”; AI phrasing; AIVOCAB; “align with”; avoidance of basic copulatives; “central”; “conceptual turning point”; “culminated”; “depicting”; “emerged from”; emojis † ; “enduring status”; “establishing a framework”; “exceptionally” † ; “focal point”; “fostering”; “here is”; “is regarded as a prominent…”; jargon; “Key X” header; LLM-typical words like “underscores”; “maintains a presence”; nonsensical language; overly consistent vocabulary † ; overuse of “referred to” † ; “perfect”; “phenomenal” † ; “pioneering † ”; “pristine” † ; random adjectives; “Remains a central theme”; “revealed”; “rich tapestry”; “showcasing”; “signalling an emphasis”; “stands as a testament”; strange wod choices † ; Subsequently; suspicious AI lexica; “the novel marked”; “they state”; too many adjectives † ; “traced to”; typical llm verbiage/vocab; “underscored”; use of word “key”; use of word “pivotal”; use of word “signify”; used “highlight” as a verb; Used word “emphasizing”; Uses AI phrase “reflects”; Uses AI phrase “significant”; Uses AI phrase “transition”; uses wording such as “highlighting”; vocab distro typical of LLMS; vocabulary co-occurrence; “whilst” † ; WP:AIVOCAB; WP:AIWORDS; WP:SAID † |
| Orthography (13) | AIBOLD; curly quote marks; em dash; excessive dashes; LLM Looking Bolding; punctuation; reference markers in plaintext † ; spaces; spaces after sentences and citations; title case; treating article title as proper noun; WP:AIBOLD; WP:AIDASH |
| Structure (66) | AILIST; AILIST/AIBOLD combo; AITABLE; avoidance of is/are phrases; Bolding of random terms; bulleted list; canned list formatting; categories; de-tag † ; deprecated lang templates † ; excessive sectioning † ; formatting; formatting irregularities † ; grammar and use of commas; header irregularities; headings; Infobox is a liable instead of an infobox template † ; inline external links † ; lead structure as standalone entity † ; length † ; List styling looks AI; listification; Lists of threes; LLM Looking Word Chaining; malformed AfC template † ; markdown † ; MARKDOWN † ; markdown code blocks; markdown dividers † ; mechanically consistent sentence structure; negative parallelism; non-Wiki markup † ; nonexisting templates, † ; nonsensical use of wiki template † ; nonstandard formatting † ; nonstandard markdown † ; outline structure about challenges † ; outline structure about future prospects † ; Outline Structure looks AI; overuse of lists † ; parallelisms; passive voice † ; reference URLs were formatted as Markdown † ; repeated words; rule of three; sentence structure; Signs of AI writing; strange stub tag † ; strangely formatted citations † ; structural signs; suspicious link style; syntax; template structure; the text is wrapped in a markdown code block † ; thematic breaks; undue list styling † ; unnecessary bolding † ; unusual use of tables † ; weird formatting † ; WP:AILIST; WP:AIMARKDOWN; WP:AIPARALLEL; WP:AITABLE; WP:AITITLECASE † ; WP:MARKDOWN; WP:RO3 |
| Quality (39) | a lot of words about not much; AIDISCLAIMER † ; Busy assortment of subheadings † ; canned emphasis on association † ; canned emphasis on identification; canned emphasis on legacy † ; canned emphasis on reportage; canned emphasis on significance † ; didactic disclaimer † ; editorializing tone † ; elaborate headings, † ; Florid prose † ; flowery † ; formulaic prose; general tone and theme † ; hedging † ; may be LLM revising specifically; Meandering † ; non neutral tone † ; not using encyclopedic tone † ; Opinionated adjectives † ; overly complex sentences † ; overstated language † ; Overuse of descriptive language † ; passive language; peacock phrasing † ; promotional tone † ; strange styled writing † ; suspicious AI style; technical evidence but unclear; tone † ; too enthusiastic † ; tortured AIATTR; vague language † ; verbose prose † ; weird prose † ; wordy † ; WP:AIDISCLAIMER † ; writing that is suspiciously sophisticated † |
| Error (13) | broken templates; contain the +1; deprecated infobox parameters; extraneous article titles; grammatical errors; harv/sfn citation error; inconsistencies; oaicitation; OAICITE; random characters in a portion of a sentence; references that redirect; WP:AIFICTREF; WP:OAICITE |
| Circular (8) | grammar; High density of AI vocabulary words; LLM-like language; prose indicators; prose reminiscent of an essay/thesis written by a language model; vocab/syntax distro typical of LLMS; vocabulary; writing style † |
| Sub-band ( ) | Labels |
|---|---|
| Inaccuracy (46) | AIREDCAT; bogus AI citation; Bogus references; content describing things that don’t exist; dab links; DOI of reference is invalid; dubious references; duplicate; fabricated source; failed verification of citations; fake citations; Fake quotes; false references; faux footnotes; Faux inline references; fictitious references; hallucinated categories; hallucinated citations; hallucinated claims; hallucination; hallucination, (main clause) , (participle modifier) structure; Inconsistency between text and sources † ; Incorrect facts; ISBNs have invalid checksums; Issues with sources; link error (404); non-existent categories; nonsenical reference; poor text-source integrity; red cats; REDCAT; Redlinked categories; redlinks; ref irregularities; reference returns 404 error; references; references don’t don’t work; references don’t exist; References have access date issues; soruces citing wrong publication; Sources that are summarised incorrectly; too much citation errors † ; Trailing ‘‘‘ from a Markdown codeblock; unsourced article spam; WP:AIREDCAT; WP:RSPLLM |
| Inappropriate (62) | adding unsupported conten † ; AIASSOCIATION † ; AIATTR † ; AILEGACY † ; AITREND † ; AIWEASEL † ; bizarre usage of literal translations of hashtags † ; book/journal references without page numbers † ; circular references † ; citation to wiki mirror † ; didactic source disclaimer variant † ; disclaimer on lack of sourcing (personal life variant) † ; disclaimers on source usage † ; graybles † ; incorrect use of ref tags † ; Irrelevant content † ; No sources † ; Overattribution † ; Overemphasis † ; overemphasis on arbitrary statements † ; overgeneralization † ; POV-pushing † ; ref to grokipedia † ; refs just go to the main pages of websites and not specific articles † ; seems very SYNTHy † ; source misrepresentation † ; specific quote not explained † ; speculation † ; Strange commentary † ; superficial editorializing † ; too many inline links † ; too much focus on random things † ; undue emphasis on association † ; undue emphasis on attribution † ; undue emphasis on being cited † ; undue emphasis on broader context † ; undue emphasis on broader entities † ; undue emphasis on broader stuff † ; undue emphasis on broader trends † ; undue emphasis on entity associations † ; undue emphasis on existence of coverage † ; undue emphasis on identification † ; undue emphasis on importance † ; undue emphasis on legacy † ; undue emphasis on media identification † ; undue emphasis on significance † ; undue emphasis on superficial analyses † ; undue emphasis on symbolism † ; undue emphasis on theme editorializing † ; undue in-text emphasis on ndependent coverage † ; undue in-text emphasis on notability † ; undue speculation † ; unnecessary weight on things like FEIN † ; vague overgeneralisations † ; vague overgeneralization of sources † ; vague unlinked citations † ; WP:AIATTR † ; WP:AICITESTYLE † ; WP:AILEGACY † ; WP:AITREND † ; WP:CIRCULAR † ; year-per-year press coverage-style bibliography † |
| Quality (16) | AIPUFFERY † ; AISUPERFICIAL † ; cruft † ; dramatic headings and subheadings † ; Fake Profundity † ; long tracts of text with few citations † ; promotionality † ; puffery † ; repeted information † ; Superficial analysis † ; typical of LLM context wavering † ; vague and superficial analysis † ; vague attributions of opinion † ; vague generalizations † ; WP:AIPUFFERY † ; WP:SUPERFICIAL † |
| Remnant (24) | AI markup bug; “article title” header; byte sequences or unicode encoding issues (turn0search0); chatbot block and a specific byte sequence; Clear artifacts of the LLM response; copypasted plaintext citation markers; file name includes LLM marker; infobox params; infobox placeholders; lack of a preceding space; LLM commentary; LLM params; non-boilerplate infobox placeholder params † ; Noticeably collaborative communication; Obvious self-referential AI-talk; placeholder text; plaintext footnote markers; string added twice; treating article title as surrogate search query; truncation; “turn0search8” bug; unicode characters; UTM links with a LLM referrer; WP:AIPLACEHOLDER| |
| Sub-band ( ) | Labels |
|---|---|
| Behavior (40) | account may have been shared between users; admitted LLM use; AfD bludgeoning; article is almost entirely written by one editor; chronic disruptive editing; contributed in massive chunks; copyright violations; Declared AI use; disclaimer about lack of sourcing; disclaimer about varying details; edit summary; English CIR issue; Harassment; humanized/ rewritten; LLM paste check; LLMPROD; marked difference in writing style of user; Mass AI generated articles; Mass AI generated comments; may be multiple authors; misleading description (e.g. edit summary); no edit summaries; overly smooth content; Pattern of AI use from a particular user; Persistent IDHT usage; physical signs (e.g. creation time or edit pattern); poor review; problematic page creations; rapid edits; removed when draftifying; revert; typical non-denial denials; user communication; user filter log; without edit summaries † ; WP:AISUMM; WP:AISUMMARY; WP:G15; WP:LLMPRVOBJ; WP:WWT |
| Trait (21) | affiliated with OKA; blocked as AI agent; blocked user; COI; confessed AI use; created by AI agent; no acknowledgement of warnings; persistent creation of non-notable subjects; probably LLM-assisted; Repeated ECR vios in the WP:CT/CASTE space; sockpuppets; TA temporary account on wiki; translation from organization that solicits AI translating; User Experienced; User Newcomer; user or editor sounds suspicious; User Student; Wikipedia:Sockpuppet investigations/; WP:DE; WP:MEAT; WP:V |
| Sub-band ( ) | Labels |
|---|---|
| External (28 labels) | |
| Authority (26) | AISIGNS; Contained G15 eligible user communication; Item from AI Cleanup Noticeboard; item from noticeboard; Special:AbuseLog; too many AISIGNS to list; Wikipedia:Administrators’_noticeboard; Wikipedia:AI_noticeboard; Wikipedia:Signs of AI writing; Wikipedia:WikiProject_AI_Cleanup/Noticeboard; WP:AI noticeboard; WP:AIN; WP:AINB; WP:AISIGNS; WP:AITELLS; WP:AN; WP:ANI; WP:ELBODY; WP:LLMISMS; WP:LLMN; WP:LLMPROD; WP:LLMPRV; WP:LLMSIGNS; WP:NEWLLM; WP:NOLLM; WPLLLMN |
| Tools (2) | Rates high on LLM detection tool; scanner identification |
| Vibes (30 labels) | |
| Circular (24) | …; AI slop; AI style; AI-esque; AIspeak; and so on; Blatant AI signs; chatbot-esque; contains common tells of LLM writing; etc; etc etc; etc etc etc; has many AI traits; historical indicators of older LLM output; hodge-podge of AI generated sentences; is just straight AI chatbot output; LLM-like construction; Looks like particular version of model; Looks like signs of a particular LLM; obvious; Points to LLM of a particular year; suspicious text; textual indicators of AI output; title is obviously LLM input |
| Inappropriate (2) | odd † ; odd formatting † |
| Rank | Label | Uses | Level-1 band | % of reasons |
|---|---|---|---|---|
| 1 | WP:AISIGNS | 2,445 | External | 40.4 |
| 2 | etc | 2,053 | Vibes | 34.0 |
| 3 | Wikipedia:AI_noticeboard | 1,801 | External | 29.8 |
| 4 | vocab distro typical of LLMS | 1,469 | Textual features | 24.3 |
| 5 | Pattern of AI use from a particular user | 1,309 | Author | 21.7 |
| 6 | Points to LLM of a particular year | 993 | Vibes | 16.4 |
AI-Generated Text Detection
AI-generated text detection develops methods to distinguish human-written, machine-generated, and AI-edited text.
See all 185 papers