Detecting communicative meaning in dialogue requires modeling not only lexical content but also the participant-role structures through which sentiment, emotion, politeness, and formality (SEPF) are expressed. Existing approaches rely on contextual language representations while leaving semantic roles implicit. We introduce a role-aware transformer architecture that maps the four SEPF dimensions to theory-grounded Semantic Role Labeling (SRL) configurations, incorporating these configurations directly into the transformer's self-attention mechanism. Role-aware attention models non-directional internal states (sentiment and emotion), while directional role-aware attention captures initiator-to-recipient relations for external cues (politeness and formality). The resulting role-aware representations are fused with contextual embeddings through a learned gate that adaptively weights how much role information each input contributes, rather than appending semantic roles as an auxiliary feature vector. Across four benchmark datasets, the framework yields statistically reliable gains over strong BERT-based baselines for emotion, politeness, and formality, with gains concentrated in theoretically predicted cases such as low-frequency, complex emotions and highly polite expressions; sentiment improvements are marginal and non-significant, suggesting that sentiment is conveyed more through lexical choice than participant-role structure. Additional evaluation on real-world data demonstrates that the framework improves both predictive performance and model traceability. These results demonstrate that linguistically grounded modifications to transformer self-attention improve the detection of communicative meaning, generalize beyond benchmark datasets to real-world mental-health data, and yield traceable, role-grounded explanations that extend beyond predictive performance to interpretability.
Figures & tables
Figure 1: Lexical and role-aware are compared using a politeness example. Lexical approaches score cue words (e.g., “Could,” “possibly”), leaving participant structure implicit. REFLECT models the interaction from initiator (ARG0) to recipient (ARG2), with mitigation modifiers (ARGM) softening the directive. Here, a politeness score (3.49) is predicted.
Figure 2: REFLECT uses complementary attention mechanisms for internal-state detection (top) and external-cue detection (bottom). For internal states, the global [CLS] query attends over pooled role vectors defined by the theory-grounded role mapping. For external cues, a source-role query (ARG0) attends over target-role positions to encode directional interactions between participants. Both branches fuse the role-aware representation with the BERT representation for prediction.
Model
Sentiment
Emotion
Politeness
Formality
Macro- F1
Macro- F1
Pearson’s r
F1
Baseline Models
BERT (linear)
0.6419
0.4594
0.6121
0.9342
BERT (MLP)
0.6594
0.4373
0.6222
0.9225
SRL-informed Models
Gated fusion, linear
0.6594
0.4803
0.6412
0.9480
Table 1: SRL-informed architectures outperform baseline models across the four communication tasks. Sentiment and emotion report macro- F1 , politeness reports Pearson’s r , and formality reports F1 . Best results are shown in bold. McNemar’s tests indicate statistically significant improvements for emotion and formality, but not sentiment.
Model
Pearson’s r
p -value
Baseline Models
BERT (linear)
0.5687
3.37×10−161
BERT (MLP)
0.5529
1.16×10−150
SRL-informed Models
Gated fusion, linear
0.5254
1.29×10−133
Gated fusion, MLP
0.5526
1.75×10−150
Table 2: Model performance on the EMA dataset. Feature concatenation yields the strongest performance. Pearson’s r is reported, with the best result in bold.
Politeness (range 0 to 4 ) Utterance: “Could you possibly send me the link?” Predicted: 3.49
No SRL
Input: Utterance + score
LLM Explanation: The use of the modal verb “Could” combined with the conditional phrasing “you possibly” softens the request, and the interrogative form signals deference; these cues may be interpreted as indicating courteous intent.
LLM Explanation: The utterance frames the agent (you) as the potential performer of the action (send) toward the recipient (me) with the object (the link). The presence of the modal “Could” and the adverb “possibly” softens the directive, which may be interpreted as a polite request, consistent with the high politeness score.
Table 3: Post-hoc explanations generated with and without SRL grounding are compared using a politeness example from the four communication tasks. The No SRL explanation identifies lexical and grammatical features. The With SRL explanation refers to the same role components used by the model’s role-aware attention pattern, illustrating that SRL-grounded explanations remain tied to REFLECT’s role-aware components used during prediction.
Prior work suggests that sentiment can often be captured by approximately linear directions in LLM activation spaces, but a single direction may not fully capture sentiment representations. In natural communication, sentiment is shaped not only by polarity but also by usage factors, such as tone and audience adaptation. We test whether these factors systematically modulate sentiment representations beyond a shared sentiment direction. We construct a controlled paired dataset that holds event content fixed while varying sentiment polarity and usage factors, and analyze Llama, Mistral, and Gemma. We identify a shared sentiment direction, remove it, and test the residual structure through erasure and generation-time tone steering. Across models, the shared direction is robust (median cosine 0.953-0.975), yet removing it leaves 0.833-0.909 of the original positive-negative representation-difference norm. The residuals contain compact, reproducible usage-conditioned structure. Targeted erasure weakens held-out usage metrics more than random and label-shuffled controls. On Llama, outputs steered along residualized tone components are preferred in 92.8% of blind target-tone comparisons while preserving the requested sentiment polarity in 98.7% of evaluated outputs.
Hongfei Du, Jiacheng Shi, Yanfu Zhang +2
Department of Computer Science, William & Mary, Williamsburg, VA, USA
Understanding how linguistic structure emerges in language models is central to interpreting what these systems learn from data and how much supervision they truly require. In particular, semantic role understanding ("who did what to whom") is a core component of meaning representation, yet it remains unclear whether it arises from pre-training alone or depends on task-specific fine-tuning. We study whether semantic role understanding emerges during language model pre-training or requires task-specific fine-tuning. We freeze decoder-only transformers and train linear probes to extract semantic roles, using performance to infer whether role information is already encoded in pre-training or learned during adaptation. Across model scales, we find that frozen representations contain substantial semantic role information, with performance improving but not fully matching fine-tuned models. This indicates partial but incomplete emergence from pre-training alone. We show that semantic role structure emerges from language modeling objectives, but its internal implementation shifts toward more distributed representations as model scale increases.
Large speech language models have demonstrated strong capabilities in unified cross-modal understanding and generation, yet paralinguistic cues, especially emotion, remain difficult to preserve. Existing systems typically rely on entangled acoustic representations, which allow the underlying language model to depend excessively on recovered lexical content instead of grounding its behavior in acoustic-prosodic evidence. We address this limitation with EMODE, an emotion-aware speech language model built around \textbf{Dynamic Para-Semantic Experts (DPSE)}. DPSE decomposes continuous speech features into semantic and paralinguistic pathways, routes them dynamically, and fuses them before integration into the language model. To turn this structural decomposition into functional specialization, EMODE is trained with a three-stage curriculum consisting of semantic warm-up, paralinguistic activation, and joint refinement, guided by Orthogonal Expert Guidance (OEG), Semantic-to-Acoustic Alignment (SAA), and Gating Diversity Regularization (GDR). Experiments on SER test, empathetic response evaluation, and the newly constructed bilingual MEPA benchmark show that EMODE improves the balance between lexical fidelity and emotional sensitivity, strengthens affect-grounded response generation, and exposes the value of explicit para-semantic factorization for robust cross-corpus emotion understanding.