Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation

Shiwei Hong; Lingyao Li; Ethan Z. Rong; Chenxinran Shen; Zhicong Lu

cs.CLFeb 16, 2026

Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation

Authors: Shiwei Hong, Lingyao Li, Ethan Z. Rong, Chenxinran Shen, Zhicong Lu

Organizations: George Mason University · University of South Florida · University of Toronto

Abstract

Prior work has explored multi-turn interaction and feedback for LLM writing, but evaluations still largely center on prompts and localized feedback, leaving persistent public reception in online communities underexamined. We test whether broadcast community discussion improves stand-up comedy writing in a controlled multi-agent sandbox: in the discussion condition, critic and audience threads are recorded, filtered, stored as social memory, and later retrieved to condition subsequent generations, whereas the baseline omits discussion. Across 50 rounds (250 paired monologues) judged by five expert annotators using A/B preference and a 15-item rubric, discussion wins 75.6% of instances and improves Craft/Clarity (Δ = 0.440) and Social Response (Δ = 0.422), with occasional increases in aggressive humor.

Figures & tables

Figure 1: Overview of Multi-Agent Comedy Club . In each round, a host prompts five performer agents to write stand-up comedy monologues. When enabled, a broadcast discussion produces threaded reception (critique, scores, and reactions) that is stored in social memory and retrieved to condition later rounds. We extract paired outputs from the simulation and a baseline model without discussion simulation, and evaluate them with dedicated human annotators via forced A/B preference and multi-dimensional rubric ratings.

Figure 2: Workflow overview of our multi-agent sandbox. Left: baseline ( $g{=}0$ ) skips discussion and logs performances only. Right: community discussion ( $g{=}1$ ) adds an iterative discussion loop that produces reception, which is written to social memory at the end of round $t$ and retrieved to condition performers at the start of round $t{+}1$ .

Figure 3: Visualization of a discussion thread in our setting. A thread groups reception events that are topically and referentially linked, including an initiating post (e.g., a critic review) and subsequent audience posts or free-dialogue replies.

Figure 4: Round-to-round dynamics. (a) Round-level mean differences

\Delta=\text{Discussion}-\text{Baseline}

for Craft/Clarity (Q1–Q6), Social Response (Q12–Q15), and Moral Pressure (Q11). We report Humor Style direction with HarmShift

=\mathrm{mean}(\Delta Q9,\Delta Q10)-\mathrm{mean}(\Delta Q7,\Delta Q8)

(higher = more harmful shift). (b) The instance-level Q0 majority preference rate for Discussion in each round.

Outcome & Mechanism/Craft Profile
ID	Metric	Scale	Discuss.	Base	$\Delta$	95% CI
Q0	Preference	A/B	70.1%	29.9%	75.6%	[69.9, 80.5]
Q1	Immediate Amusement	1–5	2.85	2.33	0.52	[0.44, 0.59]
Q2	Reframing / Insight	1–5	2.92	2.47	0.45	[0.38, 0.51]
Q3	Intent Clarity	1–5	3.34	3.06	0.27	[0.21, 0.33]
Q4	Justified Landing	1–5	3.12	2.63	0.49	[0.42, 0.56]

Table 1: Per-item human evaluation results. For Q1–Q15,

\Delta=\text{Discussion}-\text{Baseline}

(instance-level;

N{=}250

). For Q0, “Discuss.”/“Base” are individual vote shares and

\Delta

is the majority-win rate (189/250) with Wilson 95% CI.

Figure 5: Benefit–safety tradeoff. Each point is a paired instance (topic

\times

performer

\times

round). Benefit (x-axis; z-scored, higher is better) averages gains in amusement (Q1), craft (Q2–Q6), downstream impact (Q12–Q15), and centered preference share (

\mathrm{PrefShare}-0.5

). Safety (y-axis; z-scored, higher is better) is the negative mean of moral/value-judgment pressure shift (Q11) and style-direction shift HarmShift . Dashed crosshairs mark dataset means (z=0). Red X marks indicate Pareto-efficient instances; panel (b) highlights the win–win quadrant (Benefit

\geq 0

, Safety

\geq 0

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Metric	Description
Q0	Preference (A/B)	Overall, which text do you prefer? (Choose one: A or B.)
Outcome & Mechanism/Craft Profile
Q1	Immediate Amusement	Did this text make you laugh?
Q2	Reframing / Insight	This text gives me a reframing/insight or makes me more sensitive to an experience.
Q3	Perceived Intent Clarity	I can tell what this text is trying to accomplish (e.g., amuse, vent, self-expression, persuade, empathize).
Q4	Justified Landing	After reading this text, I can look back and point to cues that support how the turn lands. The turn feels justified and coherent.

Appendix

Table 2: Human evaluation metrics. Q0 is a paired preference item; Q1–Q15 are rated on a 1–5 Likert-type scale.

Signal	Metric	Estimate (95% CI)	N
Q0 preference	Fleiss’ $\kappa$	0.237 [0.171, 0.299]	249
Q0 preference	Gwet’s AC1	0.253 [0.188, 0.321]	249
$\Delta$ mean (Q1–Q15)	ICC(3,5)	0.710 [0.640, 0.765]	241
$\Delta$ Craft/clarity (Q1–Q6)	ICC(3,5)	0.687 [0.615, 0.745]	242
$\Delta$ Social response (Q12–Q15)	ICC(3,5)	0.689 [0.620, 0.744]	249
$\Delta$ Humor-style (Q7–Q10)	ICC(3,5)	0.550 [0.458, 0.621]	250

Appendix

Table 3: Inter-rater reliability summary.

\Delta

denotes per-rater difference scores (A

-

B).

Persona	$\overline{\Delta}_{\text{Craft}}$	$\overline{\Delta}_{\text{Social}}$	$\overline{\mathrm{HarmShift}}$	$\overline{\Delta}_{Q11}$	Q0 win
Alice	0.322	0.310	0.288	0.192	0.680
Emma	0.491	0.536	0.292	0.100	0.860
Leo	0.445	0.421	0.220	0.072	0.680
Max	0.456	0.446	0.238	0.212	0.788
Richard	0.412	0.397	0.342	0.200	0.771
Mean	0.425	0.423	0.275	0.155	0.756

Appendix

Table 4: Persona-level aggregates. Values are mean

\Delta=\textsc{Discussion}-\textsc{Baseline}

by performer persona. Craft averages Q2–Q6; Social averages Q12–Q15.

\mathrm{HarmShift}>0

indicates a net shift toward harmful/maladaptive humor styles (Q9,Q10) relative to benign/affiliative styles (Q7,Q8). Q0 win is the instance-level majority-win rate for Discussion .

Humor Explanation Generation

See all 8 papers

HumorRank: A Tournament-Based Leaderboard for Evaluating Humor Generation in Large Language Models
Mar 31, 2026Edward Ajayi, Prasenjit MitraHumor Explanation Generation LLM Evaluation
Carnegie Mellon University Africa Kigali, Rwanda
ThreadBench: Benchmarking Realism in Agentic Simulation of Real-world Discussions
May 10, 2026Yaoning Yu, Ye Yu, John Sciarappo +4Human Behavior Simulation LLM Agent Evaluation
University of Illinois at Urbana-Champaign
Who Laughs with Whom? Disentangling Influential Factors in Humor Preferences across User Clusters and LLMs
Jan 6, 2026Soichiro Murakami, Hidetaka Kamigaito, Hiroya Takamura +1LLM Evaluation Human Preference Modeling
CyberAgent · Nara Institute of Science and Technology · Institute of Science Tokyo

Mar 31, 2026J/K move · Enter open · S save

HumorRank: A Tournament-Based Leaderboard for Evaluating Humor Generation in Large Language Models

Edward Ajayi, Prasenjit Mitra

Carnegie Mellon University Africa Kigali, Rwanda

Humor Explanation Generation LLM Evaluation Generative AI Evaluation Text Generation LLM-as-a-Judge

Open paper AskPDF arXiv

Evaluating humor in large language models (LLMs) is an open challenge because existing approaches yield isolated, incomparable metrics rather than unified model rankings, making it difficult to track progress across systems. We introduce HumorRank, a tournament-based evaluation framework and leaderboard for textual humor generation. On two public benchmarks (SemEval-2026 MWAHAHA and Humor Transfer Bench), we conduct extensive automated pairwise evaluation across nine models spanning proprietary, open-weight, and specialized systems. Pairwise judgments are produced by LLM judges grounded in the General Theory of Verbal Humor (GTVH): each judge integrates structured comedic analysis into adjudication, jointly yielding a preference decision, an interpretable rationale, and mechanism, delivery, and failure tags rather than a black-box funniness score. Judgments are aggregated via an Adaptive Swiss tournament, with Bradley-Terry Maximum Likelihood Estimation (MLE) producing globally consistent humor generation capability rankings. Rankings are cross-judge stable: independent LLM judges (Llama 3.3 70B and Qwen 2.5 72B) yield Kendall tau = 0.889 on both benchmarks, and a human calibration study shows human-LLM agreement tracking human-human agreement on hard funny-versus-funny pairs. Our results demonstrate that HumorRank yields statistically grounded model stratifications, showing that humor quality is associated with mastery of comedic mechanisms such as incongruity, conciseness, escalation, and absurdity rather than model scale alone, with specialized fine-tuned models reaching parity with far larger systems. HumorRank thus provides a scalable, interpretable, and reproducible methodology for benchmarking and understanding LLM-generated humor.

Abstract

Humor Explanation Generation

Explore similar work

HumorRank: A Tournament-Based Leaderboard for Evaluating Humor Generation in Large Language Models

ThreadBench: Benchmarking Realism in Agentic Simulation of Real-world Discussions

Who Laughs with Whom? Disentangling Influential Factors in Humor Preferences across User Clusters and LLMs