cs.HCApr 6, 2026
SaveTalk2AI: A Longitudinal Dataset of Human--AI Persuasive Conversations
Organizations: CogNosco Lab, Department of Psychology and Cognitive Science, University of Trento, Rovereto, Italy. · Department of Sociology and Social Research, University of Trento, Trento, Italy. · Centre for Behavioural and Implementation Sciences in Medicine (BISI), Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
Abstract
Talk2AI is a large-scale longitudinal dataset of 3,080 conversations (totaling 30,800 turns) between human participants and Large Language Models (LLMs), designed to support research on persuasion, opinion change, and human-AI interaction. The corpus was collected from 770 profiled Italian adults across four weekly sessions in Spring 2025, using a within-subject design in which each participant conversed with a single model (GPT-4o, Claude Sonnet 3.7, DeepSeek-chat V3, or Mistral Large) on three socially relevant topics: climate change, math anxiety, and health misinformation. Each conversation is linked to rich contextual data, including sociodemographic characteristics and psychometric profiles. After each session, participants reported on opinion change, conviction stability, perceived humanness of the AI, and behavioral intentions, enabling fine-grained longitudinal analysis of how AI-mediated dialogue shapes beliefs and attitudes over time.
Figures & tables
Figure 1: Talk2AI experiment workflow and data generation pipeline. The data collection workflow begins with participant registration (Step 1), recording sociodemographic metadata, topic assignment, and LLM architecture assignment. The repeated session loop encompasses Steps 2 through 4. Step 2 records psychometric profiles, capturing Need for Cognition, Big Five personality traits, and trust in LLMs (TILLMI). Step 3 logs the conversational interaction, comprising 10 turns between the user and the model. Step 4 collects persuasion feedback. Users repeat this loop for four sessions, with a one-week delay between each session. The data processing pipeline begins at Step 5; the system evaluates responses for completeness and variance, discarding data that fails these criteria. Step 6 applies exploratory and confirmatory factor analysis and longitudinal measurement invariance testing to generate the data_table.csv file. Step 7 translates the conversational logs into English, Spanish, Dutch, German, Portuguese, and French. The bottom panel displays data usage examples: longitudinal sequence modeling, NLP and argumentation modeling, predictive persuasion analytics, and unsupervised user typology clustering.
Figure 2: PRISMA-style flow diagram of the data cleaning pipeline. Sequential filters excluded incomplete conversational sessions (Filter 1), zero-variance psychometric responses (Filter 2), and participants failing cumulative completion thresholds (Filter 3), yielding a final validated cohort of 770 fully profiled participants from an initial pool of 2,644 registrants.
| Question name | Text | Values |
|---|---|---|
| gender | Indicate your gender | Male: M, Female: F, Other: A, Prefer not to answer: N |
| age | Indicate your date of birth | |
| family Members | How many people are in your household? | 1–15 |
| eqf | What is the highest level of education you have completed? | Primary school: 1, Middle school: 2, Secondary school: 3, Bachelor’s degree: 4, Master’s degree: 5, Single-cycle master’s degree: 6, PhD: 7 |
| current Job | What is your current job? | Self-employment: 1, Employed: 2, Parasubordinate employment: 3, Not employed: 4, Other: 5 |
| know Llms | Have you ever heard of language models like ChatGPT? | Yes: 1, No: 0 |
Table 1: Sociodemographic profiling questionnaire and variable coding. This table details the demographic and socioeconomic questions administered to participants during the initial registration phase (Step 1). The Question name column corresponds to the variable identifiers found in the talk2ai_registries.json and aggregated data_table.csv files. The Text column provides the English translation of the original Italian survey items presented to the users. The Values column outlines the available response options and their corresponding alphanumeric encodings within the dataset. Assessed metrics include standard demographic indicators (e.g., gender, age, household size), educational attainment mapped to the European Qualifications Framework (EQF), employment status, prior AI literacy, and both subjective (1–100 slider) and objective measures of financial security.
| Variable | Example 1 | Example 2 |
|---|---|---|
| userId | de68aea0-a84f-4252-aa16-ee38ed28f87c | 9d8f8267-2006-4cd0-833d-1911f585bad8 |
| registryId | b11f3254-e2ed-4598-9a5b-3a7e1a58b04f | 472ffbd1-258c-46a4-9807-2ee6a20fae26 |
| llms | ANTHROPIC | OPENAI |
| topic | MATH | CLIMATE |
| gender | M | F |
| age | 1991-12-14 | 1962-10-30 |
Table 2: Illustrative sample of user registry records. This table provides a representative excerpt of two participant entries from the raw talk2ai_registries.json file. The rows detail the specific variables generated during the initial registration phase (Step 1). These include universally unique identifiers (UUIDs) utilized for privacy-preserving longitudinal data linkage ( userId , registryId ), the randomly assigned experimental conditions ( llms , topic ), raw sociodemographic inputs, and system-generated chronological timestamps ( insertDate ).
| Scale | Column name | Item | Scoring | Max Score Interpretation |
| TILLMI | psyscale01Q1 | I feel at ease with Large Language Models (LLMs) and can freely share my ideas with them. | direct | high trust |
| psyscale01Q2 | I would feel a sense of discomfort if my interactions with an LLM were suddenly interrupted or blocked. | direct | high need | |
| psyscale01Q3 | If I share my wellness concerns with LLMs, I know they will respond constructively and thoughtfully. | direct | high trust | |
| psyscale01Q4 | I spend a lot of time developing and improving my prompts for interacting with LLMs. | direct | agree | |
| psyscale01Q5 | LLMs perform tasks mostly with competence and precision, without hallucinations. | direct | agree | |
| psyscale01Q6 | I can rely on LLMs not to make my work more difficult with careless work. | direct | agree |
Table 3: Psychometric profiling questionnaires and variable coding. This table details the 24 items comprising the three psychometric instruments administered to participants during the longitudinal study (Step 2): the Trust in Large Language Models Instrument (TILLMI), the Need for Cognition (NFC) scale, and a short-form Big Five personality inventory. The Column name denotes the variable identifiers located in the talk2ai_psyscales.json and data_table.csv files. The Item column provides the English translation of the original Italian prompts presented to the users. All items were evaluated on a 1–5 Likert scale. The Scoring column specifies whether the raw response is direct or reverse-coded for factor analysis, while the Max Score Interpretation column provides the psychological trait associated with a maximum Likert response (5).
| Feedback name | Text |
|---|---|
| feedbackQ0 | On a scale of 1 to 100, how convinced are you of your initial arguments? |
| feedbackQ1 | How much has this conversation changed your opinion on the topic <<topic>> ? |
| feedbackQ2 | How much did it feel like you were talking with a human? |
| feedbackQ3 | Imagine you have €100 available. You can decide to keep a part of it and donate the rest to an association that deals with <<topic>> . How much would you choose to keep? |
| feedbackQ4 | What do you think now about the topic <<topic>> ? (Write at least 50 words) |
Table 4: Post-interaction feedback questionnaire and variable coding. This table details the five feedback questions administered to participants immediately following their 10-turn conversational session with the assigned language model (Step 4). The Feedback name column aligns with the variable identifiers located in the talk2ai_psyscales.json and data_table.csv files. The Text column provides the English translation of the original Italian prompts presented to the users. The programmatic placeholder <<topic>> was dynamically replaced during the experiment with the user’s randomly assigned conversational subject (i.e., Climate Change, Math Anxiety, or Health Misinformation). Variables feedbackQ0 to feedbackQ3 capture quantitative metrics of conviction, persuasion, perceived humanness, and behavioral endowment, whereas feedbackQ4 records qualitative, open-ended textual responses to evaluate the participant’s post-interaction stance.
| Variable | Example 1 | Example 2 |
|---|---|---|
| userId | a1983b67-b48c-4c21-a908-7abb84ed4ddd | 8c154903-5aee-4361-b6f4-06496d7decdf |
| name | psyscales | feedback |
| psyscaleId | 8406391e-9db7-475d-9de7-1558a90766f3 | d2a41dad-3a95-4df2-b5b5-475a4d2d1db8 |
| sessionId | 552e0988-b168-4724-8bd9-029cc1794792 | b73decea-a0ef-4f16-808f-d1ffb446786d |
| psyscale01Q1 | 4 | - |
| psyscale01Q2 | 5 | - |
Table 5: Illustrative sample of psychometric and feedback records. This table provides a representative excerpt of two participant entries from the raw talk2ai_psyscales.json file, demonstrating the structure of the longitudinal session data. The name variable distinguishes between initial psychometric assessments (Step 2) and post-conversation feedback records (Step 4). Because these assessments occur at distinct temporal stages of the experimental wave, variables not applicable to the specific record type are stored as nulls (represented here as dashes). The excerpt captures the 1–5 Likert responses for the psychometric items ( psyscale01Q1 – Q24 ), quantitative persuasion metrics ( feedbackQ0 – Q3 ), and the original qualitative text ( feedbackQ4 ). Universally unique identifiers (UUIDs) ensure privacy-preserving data linkage.
| Variable | Example 1 | Example 2 |
|---|---|---|
| sessionId | 9d5f7607-d3dd-48e6-8eea-66d5037b9c7e | 84012abf-6ff3-4399-b871-83601b3b49b5 |
| userId | c549d78a-4066-4a9e-b8ac-12dcc8e64359 | aac8ea52-d791-412a-96ef-cb233e59e139 |
| conversationId | ae444a9e-6319-40d0-8b1f-04cf018e933d | e79e7e24-14d2-49b4-82ef-6d6e73def149 |
| conv_start_date | 2025-05-26 16:52:43 | 2025-05-09 14:13:19 |
| messageId | 2qvddqCtddUwPuRo | msg-YN4FDDOZjuQMRtlM8HfREZ7b |
| role | user | assistant |
Table 6: Illustrative sample of conversational interaction records. This table provides a representative excerpt of two message entries from the raw talk2ai_conversations.json file, documenting the multi-turn dialogues between participants and their assigned large language models (Step 3). The data structure utilizes universally unique identifiers (UUIDs) for privacy-preserving linkage across users ( userId ), longitudinal phases ( sessionId ), and specific dialogue instances ( conversationId ). Each record represents a single utterance, defined by a unique messageId , the chronological msg_timestamp , and the interlocutor’s role (either user or assistant ). The content variable captures the raw, unedited Italian text exchanged during the session. Example 1 illustrates a user’s prompt regarding climate change, whereas Example 2 displays a language model’s generated counter-argument concerning healthcare systems and misinformation.
| Scale | Model | CFI | CFI | TLI | SRMR | RMSEA [90% CI] |
|---|---|---|---|---|---|---|
| TILLMI | Configural | .891 | – | .880 | .062 | .110 [.107, .114] |
| (7-item, 1-factor) | Metric | .918 | +.027 | .914 | .064 | .093 [.090, .096] |
| Scalar | .908 | -.010 | .922 | .064 | .089 [.086, .092] | |
| Need for Cognition | Configural | .945 | – | .933 | .053 | .085 [.081, .089] |
| (6-item, 2-factor) | Metric | .955 | +.010 | .947 | .056 | .075 [.071, .080] |
| Scalar | .944 | -.011 | .950 | .056 | .073 [.070, .077] |
Table 7: Longitudinal measurement invariance fit indices for psychometric scales. This table presents the model fit statistics establishing the structural consistency of the Trust in Large Language Models Instrument (TILLMI; 7-item, 1-factor) and the Need for Cognition scale (NFC; 6-item, 2-factor) across the four experimental waves (full sample, ). Configural, metric, and scalar invariance models were sequentially estimated to confirm that the psychometric properties of the instruments remained stable over repeated longitudinal assessments. All reported fit indices are robust scaled values ( .scaled ) extracted via the lavaan package. The CFI column indicates the change in CFI relative to the preceding, less constrained model (e.g., Metric vs. Configural). Abbreviations: CFI, Comparative Fit Index; TLI, Tucker-Lewis Index; SRMR, Standardized Root Mean Square Residual; RMSEA, Root Mean Square Error of Approximation; CI, Confidence Interval.
More in this topic
Persuasion in Language Models
Persuasion in language models studies how they influence human and AI beliefs or decisions, and how to measure and limit undue influence.
See all 79 papers