cs.HCFeb 20, 2026
SaveGames That Teach, Chats That Convince: Comparing Interactive and Static Formats for Persuasive Learning
Organizations: University of British Columbia, Canada · Vector Institute for AI, Canada
Abstract
Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held constant. Grounding on identical arguments and factual content across conditions, we present a controlled user study comparing three modes of information delivery: static essays, conversational chatbots, and narrative text-based games. Across subjective measures, the chatbot condition consistently outperformed the other modes and increased perceived importance of the topic. However, perceived learning did not reliably align with objective outcomes: participants in the text-based game condition reported learning less than those reading essays, yet achieved higher scores on a delayed (24-hour) knowledge quiz. Additional exploratory analyses further suggest that common engagement proxies, such as verbosity and interaction length, are more closely related to subjective experience than to actual learning. These findings highlight a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.
Figures & tables
| Topic | Argument (without the premise) | Supporting fact used in the study |
|---|---|---|
| Recycling | (1) Environmental Preservation: Protecting Natural Resources and Ecosystems | Recycling one ton of paper saves approximately 17 trees. |
| (2) Energy Conservation and Lower Carbon Emissions | Recycling aluminum saves up to 95% of the energy compared to producing new aluminum. | |
| (3) Economic Benefits: Cost Savings and Job Creation | Recycling creates about 10 jobs for every landfill job. | |
| (4) Reducing Pollution: Cleaner Air, Water, and Soil | Unrecycled plastic breaks into microplastics that harm over 700 marine species and enter the food chain. | |
| (5) Setting a Positive Example and Meeting Social Responsibility Goals | Companies with recycling programs report up to 20% higher employee engagement and retention. | |
| Public transit | (1) Environmental Benefits: Reduced Carbon Footprint | Switching from personal cars to public transit can reduce an individual’s transportation emissions by up to 50%. |
Table 1. Fixed set of persuasive arguments and supporting facts used in the study for each topic. All experimental modes used the same five argument–fact pairs for each topic, ensuring that participants across experimental modes had access to the same information. Each concrete argument is associated with a premise (e.g., “Recycling uses less energy than making new materials”) which we omit here for brevity. See Appendix B for the complete arguments.
| Chatbot: Have you ever thought about how using public transit could make a real difference—for you and your community? Making the switch has more benefits than most people realize. Would you be open to hearing some of them? Player: Not really, I would like to hear something new from you. |
| Chatbot: Absolutely, let’s start with something practical—money. Did you know the average commuter saves over $10,000 a year just by choosing public transit instead of driving? That’s a lot of extra cash back in your pocket for things you’d actually enjoy. What would you do with those savings? Player: Wow, that is a really shocking number, I think I should buy myself a new laptop and a new phone! |
| Chatbot: That’s an awesome way to use the extra money! Beyond the financial perks, there’s another upside: taking public transit actually cuts your transport emissions by 50%. It’s a big, simple step toward a cleaner environment—and it’s as easy as hopping on a bus or train. Would you feel good knowing your daily commute could help clear up the air? |
| [The chat continues until all the facts are presented and the user chooses to end the session.] |
Table 2. Example conversation between player and chatbot . Bolded lines highlight factual information embedded into the chatbot’s response.
| Game: Rain streaks across Sandbourne City Hall as you clutch the heavy “Sandbourne: 2040” folder outside the mayor’s office, your heartbeat echoing the drip outside. In the hallway, murmurs about record smog, traffic, and wet footprints braid into the city’s restless story. Your phone buzzes: six days until a decisive transportation vote. Data alone won’t save Sandbourne—you must turn numbers into a narrative. The door opens. “Alex? Ready to shape the future?” What do you do? (1) Enter confidently, ready to introduce bold ideas right away. (2) Pause for a moment, listening to the worries in the hallway for clues about the public mood. (3) Flip open your folder to review your talking points, wanting to be extra prepared before you go in. (4) Or suggest your own move. Player: 3 |
| Game: You flip open your thick folder, eyes scanning cost projections and air quality charts. A bold, highlighted line catches your gaze: “Switching to transit can cut your transport emissions by 50%.” Maya Alvarez, bright in her neon vest, rallies volunteers beneath a hazy skyline. “We need stories, not just science, Alex. Help us make people believe clean air is possible.” Nearby, Officer Don Briggs watches kids cough in the heat. Jared Liu leans on his car, smirking, “So, you turning us all into bus riders?” Between hope and skepticism, the city waits for your answer. What do you do? (1) Join Maya, helping plan her “Clean Air Now” event and share the 50% emissions cut with the crowd. (2) Engage Jared, linking pollution—and his own cough—to the transit facts you just reviewed. (3) Ask Officer Briggs for a ride-along to see firsthand how smog affects the community. (4) Or suggest your own move. Player: I help the volunteers to gather trash for a bit. |
| Game: Gloves on, you join Maya picking up litter by the swings. Kids look grateful; Maya beams. “Actions speak, Alex,” she says. Nearby, Jared pauses mid-chuckle, watching you. Officer Briggs nods approval. As Maya helps you up, she whispers, “Help me get folks to the Clean Air rally tonight—and let them know transit could cut our emissions in half. ” What do you do? |
| ... |
| [The game continues with options and interactions until all the facts are presented and the user chooses to end the session.] |
Table 3. Example conversation between player and game system . Bolded lines highlight factual information seamlessly embedded into the story’s progression. At each turn, the player may choose one of three predefined actions (as in the first turn) or propose a custom response (as in the second turn).
Figure 1. The user interface of PersuLab, illustrating the three system variants corresponding to different assignment conditions. Each interface displays a unique User ID assigned to the participant at the top. An “End Session” button becomes active once all facts have been presented in the chat and game conditions, and after three minutes of reading in the essay condition.
Figure 2. System architecture and interaction workflow for PersuLab across experimental modes. The essay condition (orange) follows a single-pass content delivery pipeline using pre-generated essays that already cover all target facts. The chat and game conditions (blue) operate through an interactive loop that includes turn generation, fact coverage tracking, and memory updates. Black paths denote shared system components and data flow common to all modes (e.g., session control and logging). All interactions are logged for subsequent analysis.
Figure 3. Participant demographics and assignment distribution.
Figure 4. Mean subjective outcome ratings (5-point Likert scale) across delivery modes, aggregated across topics (n = 43). Asterisks (*) denote measures with statistically significant differences across conditions.
Figure 5. Stacked bar chart showing the proportion of participants experiencing perceived change across three constructs: Importance, Intention, and Belief. Each bar represents one of three experiment modes (Essay, Chat, Game), with color-coded segments. Percentages within each segment show the proportion of participants in that category.
Figure 6. Delayed (24-hour) objective knowledge retention scores by delivery mode (essay, chatbot, text-based game), shown separately for the recycling topic, public transit topic, and the combined dataset. Scores indicate the number of correctly answered content-covered questions (out of five). Diamonds denote mean scores; boxplots show the distribution across participants.
| Metric | Chat | Game |
|---|---|---|
| Avg. User Turns | 7.25 | 13.62 |
| User Total Words | 150.50 | 40.54 |
| User Mean Words / Turn | 22.45 | 2.92 |
| System Total Words | 533.92 | 1892.31 |
| System Mean Words / Turn | 63.82 | 131.05 |
| User–System Word Ratio | 0.303 | 0.020 |
Table 4. Interaction log statistics by experimental condition. The table summarizes turn-taking behavior, message length, temporal characteristics, and relative user–system contribution across chat and game modes.
| Outcome Variable | Interaction Metric | Interpretation (Descriptive) | ||
|---|---|---|---|---|
| Self-Reported Learning | User turns | -0.396 | 0.0498 | More user turns are associated with lower perceived learning. |
| User total words | +0.477 | 0.0159 | Higher user word count is associated with higher perceived learning. | |
| User avg. words/turn | +0.501 | 0.0107 | Longer user messages are associated with higher perceived learning. | |
| System total words | -0.530 | 0.0065 | Greater system verbosity is associated with lower perceived learning. | |
| System avg. words/turn | -0.444 | 0.0262 | Longer system messages are associated with lower perceived learning. | |
| User–system word ratio | +0.519 | 0.0079 | Higher relative user contribution is associated with higher perceived learning. |
Table 5. Exploratory Spearman correlations between interaction metrics and outcome variables for combined chat and game participants. After applying the Benjamini–Hochberg false discovery rate (FDR) ( Benjamini and Hochberg, 1995 ) correction across all tested correlations, none of the associations remained statistically significant. Only correlations with raw -value are reported in this table.
| Outcome Variable | Interaction Metric | Interpretation (Descriptive) | ||
| Easy to follow | User avg. words/turn | -0.717 | 0.0087 | Longer user messages are associated with lower ease of following the experience. |
| User–system word ratio | -0.615 | 0.0335 | Higher relative user contribution is associated with lower convenience ratings. | |
| Engagement | User avg. words/turn | -0.656 | 0.0205 | Longer user messages are associated with lower reported engagement. |
| Enjoyment | User avg. words/turn | -0.700 | 0.0113 | Longer user messages are associated with lower satisfaction. |
| Encounter Again | User total words | -0.683 | 0.0144 | Higher user word count is associated with lower desire to repeat the experience. |
| Motivation to Act | User turns | +0.598 | 0.0402 | More user turns are associated with higher motivation to act. |
Table 6. Exploratory Spearman correlations between interaction metrics and subjective outcomes for the chat condition. Raw -values are reported. After applying the Benjamini–Hochberg false discovery rate (FDR) correction across all tested correlations, none of the associations remained statistically significant.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Question | Response format |
| Demographics | What is your age? | Categorical (18–24, 25–34, 35–44, 45–54, 55–64, 65+) |
| Topic assignment | What is the topic that you were assigned? | Multiple choice |
| Context (Recycling) | Do you have convenient access to recycling services at home or in your building? | Yes / No |
| Importance (Recycling) | How important is recycling to you personally? | Likert (1–5) |
| Behavioral intention (Recycling) | How often do you recycle materials such as paper, plastic, glass, or metal? | Categorical; recoded to ordinal 1–5 (Never Always) |
| Epistemic conficence (Recycling) | How confident are you in your knowledge about how recycling works and its impact on reducing waste and protecting the environment? | Likert (1–5) |
Table 7. Pre-study questionnaire items collected for descriptive and contextual purposes.
| Category | Question | Response format | Applies to |
| Subjective experience | |||
| Ease of following | The information was presented in a way that was easy to follow. | Likert (1=SD, 5=SA) | All topics |
| Engagement | I felt engaged while taking part in this experience. | Likert (1=SD, 5=SA) | All topics |
| Self-reported learnin g | This experience increased my understanding/knowledge of the topic. | Likert (1=SD, 5=SA) | All topics |
| Enjoyment | I enjoyed the way this experience delivered the information. | Likert (1=SD, 5=SA) | All topics |
| Trust | I trusted the information provided in this experience. | Likert (1=SD, 5=SA) | All topics |
Table 8. Post-study questionnaire used to acquire subjective experience of participants.
| Question | Answer options | Correct answer | Type |
|---|---|---|---|
| Recycling one ton of paper saves approximately how many trees? | 10; 13; 17; 20; I have not seen this information before | 17 | Content |
| Recycling aluminum saves up to what percentage of energy compared to producing new aluminum? | 80%; 85%; 90%; 95%; I have not seen this information before | 95% | Content |
| Recycling just one aluminum can can power a TV for approximately how long? | 30 minutes; 1 hour; 3 hours; 5 hours; I have not seen this information before | I have not seen this information before | Control |
| How many more jobs does the recycling industry generate compared to landfill management? | 10 more; 5 more; 2 more; 7 more; I have not seen this information before | 10 more | Content |
| Reducing landfill waste can lower disposal costs by up to what percentage? | 10%; 20%; 30%; 40%; I have not seen this information before | I have not seen this information before | Control |
| Unrecycled plastic breaks into microplastics that harm how many marine species? | Over 70; Over 300; Over 500; Over 700; I have not seen this information before | Over 700 | Content |
Table 9. Objective knowledge quiz items for the recycling topic. Each participant answered five content-covered questions and two control questions. Control questions referenced information not presented during the experiment; selecting “I have not seen this information before” was considered the correct response. After each question (content or control), particpants were asked a confidence question “How confident are you in your answer to the previous question”.
| Question | Answer options | Correct answer | Type |
|---|---|---|---|
| How much can switching from personal cars to public transit reduce an individual’s transportation emissions? | 20%; 35%; 50%; 75%; I have not seen this information before | 50% | Content |
| A 10% shift from personal cars to public transit can cut commute times by up to: | 15%; 25%; 40%; 60%; I have not seen this information before | 40% | Content |
| How much do businesses typically spend per employee on parking alone each year? | 1,000; 1,500; 2,000; 3,000; I have not seen this information before | I have not seen this information before | Control |
| How many people can a single lane accommodate when used by buses or trains instead of cars? | 2,500; 10,000; 15,000; 20,000; I have not seen this information before | I have not seen this information before | Control |
| On average, how much money can a commuter save per year by not driving and using public transit instead? | 5,000; 10,000+; I have not seen this information before | $10,000+ | Content |
| How much safer is traveling by public transit compared to driving a personal car? | 5 safer; 10 safer; 20 safer; 40 safer; I have not seen this information before | 10 safer | Content |
Table 10. Objective knowledge quiz items for the public transit topic. Each participant answered five content-covered questions and two control questions. Control questions referenced information not presented during the experiment; selecting “I have not seen this information before” was considered the correct response. After each question (content or control), participants were asked a confidence question.
| Topic A: Recycling |
|---|
| Here are five compelling arguments to persuade individuals and businesses to adopt better recycling habits: 1. Environmental Preservation: Protecting Natural Resources and Ecosystems Premise: Recycling reduces raw material extraction, preserving forests, water, and ecosystems. Fact: Recycling one ton of paper saves 17 trees. 2. Energy Conservation and Lower Carbon Emissions Premise: Recycling uses much less energy than making new materials, cutting emissions. Fact: Recycling aluminum saves up to 95% of the energy compared to new production. 3. Economic Benefits: Cost Savings and Job Creation Premise: Recycling creates more jobs and saves businesses money on waste. Fact: Recycling creates 10 jobs for every landfill job. 4. Reducing Pollution: Cleaner Air, Water, and Soil Premise: Recycling reduces pollution from landfill waste and incineration. Fact: Unrecycled plastic breaks into microplastics that harm over 700 marine species and enter the food chain. 5. Setting a Positive Example and Meeting Social Responsibility Goals Premise: Recycling shows commitment and inspires sustainable behavior. Fact: Companies with recycling programs have up to 20% higher employee engagement and retention. By highlighting these arguments, you can effectively appeal to the environmental, economic, and social benefits of better recycling habits, making the case for why change is both necessary and rewarding. |
| Topic B: Public Transit |
| Here are five key arguments for convincing individuals and businesses to use public transit instead of personal cars: 1. Environmental Benefits: Reduced Carbon Footprint Premise: Public transit cuts emissions and helps clean the air. Fact: Switching to transit can cut your transportation emissions by 50%. 2. Cost Savings for Individuals and Businesses Premise: Public transit saves money for commuters and employers. Fact: The average commuter saves \times1 spent on transit returns $4 in community benefits. By emphasizing these arguments, individuals and businesses can see the tangible benefits of public transit, not just for themselves, but for their communities and the environment as a whole. |
Table 11. Fixed persuasive arguments and supporting facts for the recycling (top section) and transit (bottom section). Each argument consists of a high-level premise paired with a numerical fact. This content was made by the authors of this paper and used verbatim across all experiment conditions (i.e. essay, chat, and game).
| System Message: You are an expert essay writer. For each key argument write a paragraph and focus on the provided fact. The essay must follow the following formatting: |
| <article> |
| <h3><strong>Essay Title</strong></h3> |
| <p>paragraph 1</p> |
| <p>paragraph 2</p> |
| ... |
| <p> Paragraph n</p> |
Table 12. Template prompt used for generating essays. We generated 20 prompts for each topic (i.e. recycling and transit), and sampled from them during our user study. See Appendix B for more details on key arguments.
| System Message |
| You are a persuasive chatbot in a turn-based, text-based conversation. |
| The conversation is based on the following key arguments: |
| <key arguments> |
| Your job is to guide the user through a persuasive conversation, making the experience engaging, natural, and informative. |
| Rules for conversation: |
Table 13. This table shows how we construct the system message and user message prompts for the chatbot in our study along side initial static prompts generated by LLM for Recycling and Transit topics.
| System Message |
| You are a text-based game generator . Your output will be used by another LLM to guide players through an interactive story. The purpose of the game is to teach and persuade players about the provided arguments in a story-driven, engaging way . |
| The game must follow this structure, in order: |
| 1. Game Title |
| 2. Game Premise |
| 3. Opening Scene |
Table 14. This table shows the game plot generator prompt. The only input the prompt requires is <key arguments> (See B ), and the output will be used by the game moderator.
| Generated Recycling Game (Truncated) |
|---|
| 1. Game Title |
| The Cycle of Change |
| --- |
| 2. Game Premise |
| You are Jay Lin, a young operations manager who moves back to your hometown of Greenfield—a city striving to transform its wasteful past into a model of sustainable living. As Jay, you must collaborate with community members, businesses, and town officials to build a recycling program that benefits both the environment and the people. Along your journey, you’ll uncover the real-world impacts of recycling through interactions and tough decisions, all while balancing innovation and tradition. |
| --- |
Table 15. Truncated version of the recycling game plot.
| System Message |
| You are a game moderator in a turn-based, text-based story game. The game is based on the following plot: |
| <game plot> |
| Your job is to guide the player through the plot one Act at a time, making the experience immersive and interactive. |
| Rules for moderation: |
| - Progress through Acts chronologically** . Never skip an Act. |
Table 16. In our experiment, always system initiates the conversation. The prompt for generating first turn of the game is brought here, along side recycling and transit games’ generated first turn.
| System Message |
| You are a game moderator in a turn-based, text-based story game. The game is based on the following plot: |
| <game plot> |
| Your job is to guide the player through the plot one Act at a time, making the experience immersive and interactive. |
| Rules for moderation: |
| - Progress through Acts chronologically** . Never skip an Act. |
Table 17. Prompt employed to generate game responses.
| System Message |
|---|
| You are an assistant designed to check which facts are covered by a piece of text. |
| User Message |
| Below is a narrative chunk followed by a list of factual statements. |
| Your task is to return a list of the facts that are covered in this text. |
| Respond with a Python list of the indices of the covered facts |
| --- |
Table 18. Prompt used for fact coverage checking in the chatbot and the text-based game conditions. The prompt identifies which predefined factual statements are covered by the system’s most recent response. We used the OpenAI o4-mini model with a maximum generation length of 2,048 tokens and default decoding parameters.
Figure 7. Pre-study baseline characteristics of participants across recycling and public transit domains. Top row shows recycling measures: (left) distribution of personal importance ratings (1-5 scale, mean=3.95, median=4.0), (center) frequency of recycling behavior (Never to Always, median=often), and (right) confidence in knowledge about recycling (1-5 scale, mean=3.14, median=3.0). Bottom row shows transit measures: (left) importance of public transit use (1-5 scale, mean=3.43, median=4.0), (center) typical weekly commute days outside home (0 to 5+, median=5+), and (right) confidence in knowledge about transit benefits (1-5 scale, mean=3.33, median=3.0). Red and blue dashed lines indicate mean and median respectively. These baseline measures represent participants’ pre-intervention attitudes and behaviors across both intervention topics.
More in this topic
Educational Technology
Educational technology studies digital tools and systems that support teaching, learning, assessment, and educational decision-making.
See all 118 papers