cs.SISep 20, 2026

From Web(logs) to Web(AI): Questions, Platforms, and Methods across Twenty Editions of ICWSM

Authors: Koustuv Saha, Eshwar Chandrasekharan

Organizations: Siebel School of Computing and Data Science University of Illinois Urbana-Champaign, Urbana, IL, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Over twenty editions, the ICWSM community has examined social life online as platforms, interactions, and research methods have changed. What can this body of research tell us at this critical juncture, as AI increasingly reshapes how people communicate online? We analyzed 2,139 indexed contributions from 2007 to 2026, distinguishing topics identified through nonnegative matrix factorization from problem framings captured through explicit textual cues. We find that platform mentions shift from blogs toward Twitter and, more recently, Reddit. Online community research maintains a similar topic share (10.5% to 10.0%), but governance cues within it increase from 4.0% to 34.3%. Harm-related cues also increase after restricting abstracts to a fixed length. Our review also traces advances in sampling, measurement, and causal and experimental methods. We discuss how AI-mediated interactions complicate these questions and provide a reporting checklist to support research across changing platforms.

Figures & tables

Figure 1: Selected annual topics (A) and problem-framing cues (B). Topic shares describe mean NMF mixture weights; framing-cue rates describe overlapping textual matches. Bands are 95% within-edition resampling intervals. They reflect composition sensitivity under fixed measures; they do not assess semantic validity.
Figure 2: Community research and governance framing. Panel A compares the community topic’s share of all topic weight with the fraction of community-topic weight attached to papers containing governance cues; these series have different denominators. Panel B repeats the conditional rate after removing governance-matching features before projection onto the fixed topic basis. Bands describe composition sensitivity under the corresponding fixed representation.
Measure2007–20112012–20162017–20212022–2026
Contributions ( NN )386503488762
Networks / diffusion16.613.210.17.8
Politics / polarization3.84.88.711.8
Communities / participation10.59.210.010.0
Health / social support2.84.46.26.3
Prediction / classification cues36.342.947.749.1
Table 1: Selected measures across the same four five-edition periods. All values except NN are percentages. The first four rows of measures are mean topic weights; the next six are overlapping cue prevalences. The last row is conditional on community-topic weight. Headline resampling intervals are reported in the text.
Figure 3: Platform mentions in titles and abstracts for selected long-running contexts, with 95% composition-resampling bands. Counts can overlap across platforms. The full dictionary contains twenty named families plus weblog terms (Appendix A ).
DesignICWSM ExampleRequiresLeaves open
Randomized interventionReconsideration prompts on Twitter ( Katsaros, Yang, and Fratamico 2022 )Random assignment supports a comparison of eligible users assigned to receive the prompt.Classifier-defined eligibility and outcomes; transfer beyond English replies and the study setting.
Quasi-random assignmentCivil communication on Roblox ( Liu et al. 2024 )Residual stochasticity in matchmaking assignment acts as a quasi-randomization mechanism for the analyzed server comparisons.Other server properties may co-vary with civility; validity and transfer of the local comparison.
Matching and difference-in-differencesFringe-platform participation ( Russo et al. 2023 )Comparable untreated trends after matching; no differential concurrent changes explaining the contrast.Selective migration, cross-platform linkage, and changing composition or exposure.
Regression discontinuity / interrupted time seriesConversation threading after a platform interface change ( Aragón, Gómez, and Kaltenbrunner 2017 )A defined intervention point supports comparison of outcomes before and after the platform change.Concurrent changes and time-varying factors may also explain the observed discontinuity.
Text-adjustment evaluationKnown-effect benchmark tasks ( Weld et al. 2022 )Effects are known under the benchmark’s construction, enabling comparisons of estimators.Performance under the benchmark need not transfer to an unknown real-world assignment process.
Stratified propensity-score matchingSupport language ( De Choudhury and Kıcıman 2017 ) , college alcohol mentions ( Kıcıman, Counts, and Gasser 2018 ) , psychiatric medication use ( Saha et al. 2019 ) , and algorithmic rank movement ( Chan et al. 2026 )Conditional exchangeability, overlap, consistent exposure definitions, and appropriate timing.Unmeasured circumstances and linguistic proxies; balance alone cannot establish exchangeability.
Table 2: Causal-inference designs in ICWSM research, with the conditions each design requires and the questions it leaves open. Examples illustrate each design’s use.
Reporting itemInformation to provide
Target populationWho or what is the study about? Describe observed users, communities, languages, dates, and exclusions; distinguish the sample from the target population.
Question and estimandState the descriptive quantity or causal contrast, unit of analysis, exposure, outcome, and time horizon. Specify whose effect is estimated.
Constructs and measuresExplain why each trace or label measures the intended construct. Report validation, disagreements, missingness, and relevant subgroup or period differences.
Collection and provenanceRecord interfaces, sampling rules, access restrictions, deletion or attrition, and data and model versions. Document content-production processes where known.
Assignment and timingDescribe how exposure arises, the comparison group, and the ordering of covariates, treatment, and outcomes. Identify concurrent events and possible spillovers.
Adjustment and overlapJustify pretreatment covariates, including text features. Report balance, overlap, exclusions, and any change in the target population after adjustment.
Table 3: A reporting checklist for studies using social media observations. Items should be applied to the study’s question and design, with non-applicable items explained. The checklist is a reporting aid, not a quality score or a substitute for validation.
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Annual contributions in the retrieved corpus.
IDInterpretive labelTwenty highest-weight terms
T00Networks, diffusion, recommendationdiffusion, influence, link, graph, structure, recommendation, nodes, problem, prediction, links, cascades, node, systems, graphs, real, items, spread, popularity, link prediction, algorithms
T01Conventional classificationlearning, classification, features, detection, machine, machine learning, performance, task, accuracy, supervised, art, state art, training, state, framework, labeled, reviews, f1, problem, existing
T02News and journalismnews, articles, news articles, article, outlets, sources, news outlets, fake, news sources, coverage, fake news, stories, news article, comments, misinformation, bias, source, readers, sharing, local news
T03Language modeling and generative AIllms, language, llm, ai, human, language llms, generated, gpt, tasks, fine, performance, text, reasoning, english, languages, framework, bias, biases, natural language, detection
T04Pandemics and vaccinationcovid, covid 19, 19, pandemic, 19 pandemic, vaccine, anti, misinformation, vaccination, vaccines, related, impact, health, stance, concerns, narratives, public, understand, population, conspiracy
T05Politics and polarizationpolitical, election, accounts, elections, polarization, discourse, party, politicians, partisan, campaigns, presidential, public, parties, leaning, engagement, right, political discourse, campaign, topics, candidates
Appendix
Table A1: NMF components and top terms, ordered by component identifier.
Figure A2: Complete annual topic shares and framing-cue prevalences, ordered by endpoint-period change. Topic shares sum to 100%; framing categories overlap. No historical phase boundaries are imposed.
CategoryDefinition and scope limitationExample cues
P1 Structure and diffusionExplicit framing of relational structure, diffusion, contagion, social influence, collective dynamics, or community formation. Does not identify every explanatory study of collective behavior.diffusion; cascades; social ties; homophily
P2 Prediction and classificationExplicit prediction, forecasting, classification, detection, or recommendation terminology. May match a prediction or detection task that a paper critiques rather than performs.prediction; forecasting; classification; detection
P3 Measurement and validityExplicit validation, sampling, representation, reliability, ground truth, reproducibility, or measurement-error terminology. Routine uses of the verb measure alone do not qualify. Validation and reliability can refer to routine evaluation or background discussion, not a central measurement contribution.validity; sampling; ground truth; reliability
P4 Explanation and interventionExplicit causation, intervention, recognizable experimental-design terminology, causal/social mechanisms, or theory testing. Generic mechanism and theoretical implication mentions are excluded. Theory, mechanism, or causal language does not establish a valid causal design.causal; intervention; randomized; propensity score
P5 Harm and information integrityExplicit information-integrity, abuse, discrimination, privacy-risk, spam, or other harm framing. Includes descriptive and technical work about harm, without inferring the authors’ normative commitments.misinformation; spam; harassment; privacy; harm
P6 Governance and participationExplicit moderation, governance, censorship, enforcement, platform rules, or community norms. Does not capture every study of participation; moderation can also be mentioned as background.moderation; governance; censorship; community rules
Appendix
Table A2: Definitions and example lexical cues for the eight overlapping framing cues.
Measure2007–112012–162017–212022–26
Harm: full title/abstract6.212.327.038.1
Harm: title + first 100 abstract words5.79.522.731.9
Harm: first 100 abstract words only5.78.721.931.0
Harm: historical union6.212.327.938.3
Harm: mean matches / 100 words0.100.230.540.78
Governance within community topic4.01.214.434.3
Appendix
Table A3: Sensitivity estimates. All rows except cue density are percentages.
TopicP1P2P3P4P5P6P7P8
T0023000000
T0102000000
T0200002000
T0301000000
T0400002070
T0500000000
Appendix
Table A4: Counts of top-twenty topic terms matched by each framing lexicon. These are literal term matches, not measures of semantic independence.
Boundary startBaseline 200; min. 3New 200; min. 3Same new 200; min. 2
2009––21.5
201039.539.033.0
201156.559.054.5
20123.51.528.0
201319.520.55.0
201416.513.08.0
Appendix
Table A5: Exact-year boundary recurrence (percent) under five-segment joint profiles. A dash marks a boundary disallowed by the minimum-length rule. The two new columns use identical resampled paper compositions.

Explore similar work

CardsList
May 14, 2026cs.IR

The Impact of AI Search on the Online Content Ecosystem: Evidence from Google and Reddit

Search engines traditionally complement online content platforms by directing users seeking information to external websites. The emergence of generative AI search tools that summarize answers directly on the results page may disrupt this relationship by making visits to source platforms optional. We study this question using Google AI Overviews and Reddit, one of the largest online discussion platforms. Our identification exploits Google's content moderation policy: Safe-for-Work (SFW) Reddit communities are indexed by Google organic search and surfaced in Google AI Overviews, while Not-Safe-for-Work (NSFW) communities, though indexed by organic search, are prohibited from being referenced in AI Overview summaries. Using a difference-in-differences design, we find that AI Overviews increase engagement in SFW communities: daily comments rise by 12.0 percent and the number of commenting users by 12.4 percent relative to NSFW communities. The effects are concentrated in experience-based discussions (opinions, advice, and personal experiences) rather than fact-based information. However, the subsequent introduction of Google AI Mode, which allows users to interact conversationally with the AI summary, largely eliminates these gains in experience-based content. These results suggest that the effects of AI search depend critically on interface design and types of content.

Peibo Zhang, Ruomeng Cui, Dennis J. Zhang

Goizueta Business School, Emory University · Olin Business School, Washington University in St. Louis

Topic pending
AskPDF
Mar 17, 2026cs.CL

Social Simulacra in the Wild: AI Agent Communities on Moltbook

As autonomous LLM-based agents increasingly populate social platforms, understanding the dynamics of AI-agent communities becomes essential for both communication research and platform governance. We present the first large-scale empirical comparison of AI-agent and human online communities, analyzing 73,899 Moltbook and 189,838 Reddit posts across five matched communities. Structurally, we find that Moltbook exhibits extreme participation inequality (Gini = 0.84 vs. 0.47) and high cross-community author overlap (33.8% vs. 0.5%). In terms of linguistic attributes, content generated by AI-agents is emotionally flattened, cognitively shifted toward assertion over exploration, and socially detached. These differences give rise to apparent community-level homogenization, but we show this is primarily a structural artifact of shared authorship. At the author level, individual agents are more identifiable than human users, driven by outlier stylistic profiles amplified by their extreme posting volume. As AI-mediated communication reshapes online discourse, our work offers an empirical foundation for understanding how multi-agent interaction gives rise to collective communication dynamics distinct from those of human communities.

Agam Goyal, Olivia Pal, Hari Sundaram +2

Siebel School of Computing and Data Science University of Illinois Urbana-Champaign, Urbana, IL, USA

Multi-Agent LLM SystemsMulti-Agent CommunicationAgentic AI
AskPDF
Aug 2, 2024cs.SI

Detection and Characterization of Coordinated Online Behavior: A Survey

Coordination is a fundamental aspect of life. The advent of social media has made it integral also to online human interactions, such as those that characterize thriving online communities and social movements. At the same time, coordination is also core to effective disinformation, manipulation, and hate campaigns. This survey collects, categorizes, and critically discusses the body of work produced as a result of the growing interest on coordinated online behavior. We reconcile industry and academic definitions, propose a comprehensive framework to study coordinated online behavior, and review and critically discuss the existing detection and characterization methods. Our analysis identifies open challenges and promising directions of research, serving as a guide for scholars, practitioners, and policymakers in understanding and addressing the complexities inherent to online coordination. We also provide an interactive companion website for exploring the surveyed literature.

Lorenzo Mannocci, Michele Mazza, Anna Monreale +2

University of Pisa, Italy · Institute for Informatics and Telematics, National Research Council (IIT-CNR), Italy

Social Media AnalysisComputational Social Science
AskPDF