Personalized Language Models for One-to-Many Private Communications
Organizations: The University of Queensland
Abstract
We study relationship-specific continual reply prediction for one-to-many private communication: a system predicts an author's reply to a recurring contact and updates its personalization state when the actual reply becomes available. We propose TRACE (Temporal Relationship-Aware Communication Evolution), which uses author-level lexical habits, contact-specific wording, and temporal expression estimates to guide a frozen language model, with a feedback-driven policy controlling revision. On 216 test emails from eight Enron users, all four main TRACE configurations achieve higher mean BLEU, ROUGE-L, and BERTScore and lower mean Reply Style MSE than Base LLM and Base RAG. Component ablations show mixed effects and do not establish statistically reliable improvements in Reply Style MSE after multiple-comparison correction. More accurate state forecasts and larger revision budgets do not consistently improve reply-level metrics. These exploratory findings expose the challenge of turning evolving relationship-specific expression estimates into useful generation guidance.
Figures & tables
| Method | State Prediction MSE | BLEU | ROUGE-L | BERTScore | Reply Style MSE | ||
| Base LLM | 1 | – | – | 0.785 0.599 | 9.282 2.446 | 84.188 0.775 | 0.09644 0.01436 |
| Base RAG | 1 | – | – | 1.837 1.378 | 12.935 5.168 | 86.060 0.862 | 0.05099 0.01097 |
| TRACE (reference) | 2 | 3 | 0.02905 0.00680 | 2.710 3.297 | 13.414 4.892 | 86.601 0.920 | 0.04349 0.01051 |
| TRACE (more revisions) | 3 | 3 | 0.02905 0.00680 | 2.921 3.297 | 14.332 6.009 | 86.650 0.863 | 0.04415 0.01023 |
| TRACE (short window) | 2 | 2 | 0.02916 0.00681 | 2.828 3.354 | 14.256 6.132 | 86.826 0.936 | 0.04571 0.01193 |
| TRACE (long window) | 2 | 5 | 0.02857 0.00654 | 2.778 3.335 | 13.753 6.625 | 86.764 0.767 | 0.04561 0.01262 |
| Method | State Prediction MSE | BLEU | ROUGE-L | BERTScore | Reply Style MSE |
|---|---|---|---|---|---|
| Full TRACE | 0.02905 0.00680 | 2.999 3.274 | 14.532 5.673 | 86.804 0.871 | 0.04195 0.01008 |
| w/o Global Lexical Summary | 0.02905 0.00680 | 2.468 3.097 | 13.460 5.592 | 86.635 0.771 | 0.04475 0.01006 |
| Shuffled Contact TF-IDF | 0.02905 0.00680 | 2.699 2.915 | 14.391 4.645 | 86.837 0.833 | 0.04234 0.00920 |
| w/o Temporal Modeling | 0.02787 0.00662 | 2.903 3.317 | 14.964 5.481 | 86.899 0.766 | 0.04211 0.01137 |
| w/o Q-learning | 0.02905 0.00680 | 2.996 3.238 | 14.200 5.959 | 86.707 0.967 | 0.04293 0.01038 |
| w/o Revision | 0.02905 0.00680 | 2.973 3.300 | 14.563 5.630 | 86.766 0.955 | 0.04269 0.01070 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Personalized Language Models
Personalized language models adapt responses to individual users’ preferences, histories, writing styles, and changing needs.
See all 99 papers