cs.CLOct 16, 2026
SaveMulti-author style detection with stylistic embeddings
Organizations: RTX BBN Technologies Cambridge MA, USA
Abstract
We investigate the use of Large Language Models (LLMs) in multi-author style change detection. Using off-the-shelf LLMs or supervised fine-tuning solely within the target domain fails to yield effective features; for this reason, we propose cross-domain transfer learning with LLM-based embeddings fine-tuned for authorship verification. We fine-tune Mistral-7B embedding model and compare it with DeBERTa, used in the state-of-the-art (SOTA) systems in PAN-25 multi-author writing style analysis challenge. Results show that DeBERTa and Mistral excel in different user-generated text scenarios. We exploit this diversity by combining both embeddings, leading to more effective style change detection.
Figures & tables
Figure 1: The proposed system. LUAR-Mistral generates embeddings from k sentence-pairs, where k is the number of sentences. This figure illustrated the single sentence case where k=1. The MLP classifier takes in both sentence embeddings and their absolute difference to decide if authorship change happens in between. The sentence embeddings come from a modified LUAR framework which replaces BERT with Linq-embeded Mistral. The framework is fine-tuned to optimize the contrastive loss of same and different authors.
| PAN-25 (Valid) | Easy | Medium | Hard |
|---|---|---|---|
| Qwen2-1.5B | 0.872 | 0.798 | 0.703 |
| LE-Mistral-7B | 0.904 | 0.834 | 0.734 |
| mE5 | 0.862 | 0.772 | 0.685 |
Table 1: Results on PAN-25 validation set adopted with LUAR framework. Sentence-level features is used. For an explanation of the easy/medium/hard conditions, please read Section 4.1 .
| Task | PAN 2025 (test) | PAN 2026 (valid) | ||||
| Model | Easy | Medium | Hard | Easy | Medium | Hard |
| Baseline Bevendorff et al. (2025) | 0.439 | 0.44 | 0.453 | 0.410 | 0.489 | 0.442 |
| Llama-3 (4-bit) ft Chen et al. (2025) | 0.461 | 0.583 | 0.484 | / | / | / |
| Claude-3.7-sonnet Schmidt et al. (2025) | 0.856 | 0.818 | 0.661 | / | / | / |
| SCL-DeBERTa Lin et al. (2025a) | 0.950 | 0.812 | 0.827 | 0.980 | 0.550 | 0.489 |
| DeBERTa-Classifier Lin et al. (2025b) | 0.958 | 0.822 | 0.829 | 1.000 | 0.633 | 0.735 |
Table 2: Results on PAN-25/26. The top table contains results with previous systems, second table is with our own systems, bottom table is with combined systems. The results for LLama-3 and Claude-3.7-sonnet are copied from Zangerle et al. (2025) , while the rest are our own reproduction. The best overall result in each column is bolded, while the best single system result is underlined.
| Genre | DeBERTa | Mistral | +k-sent |
|---|---|---|---|
| B.G. Reviews | 0.710 | 0.731 | 0.734 |
| Instruction | 0.609 | 0.686 | 0.694 |
| journalism | 0.809 | 0.780 | 0.798 |
| LIT. forum | 0.776 | 0.790 | 0.779 |
| STEM forum | 0.913 | 0.744 | 0.755 |
Table 3: F1 scores on the five HRS genres. Notations: B.G.- Boardgame, LIT. - Literature.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| PAN-25 | ||||
| Task | Set | # Doc | #Sent | #Word/Sent |
| Easy | Train | 8.4k | 48k | 18.31 |
| Valid | 1.8k | 10k | 18.42 | |
| Test | 1.8k | 10k | 18.55 | |
| Medium | Train | 8.4k | 59k | 18.91 |
| Valid | 1.8k | 13k | 19.00 | |
Table 4: Detail of the train, valid and test sets for the 3 tasks in PAN25 and PAN26, presented in number of documents, sentences and average number of word in each sentence.
| PAN-25 Easy (4 authors) |
|---|
| In general, be courteous to others. |
| Debate/discuss/argue the merits of ideas, don’t attack people. |
| Personal insults, shill or troll accusations, hate speech, any suggestion or support of harm, violence, or death, and other rule violations can result in a permanent ban. |
| Correcto. |
| They refer to them more easily as animals or a disease. |
| Only suitable to be wiped out and systematically destroyed. |
Table 5: Problem 10 of PAN-25 easy training set. Beside the stylistic changes, transition can be identified easily with words that indicate the response to the previous users – "Correct", "Exactly" and topic header – "r/politics".
| PAN-25 Medium (2 authors) |
|---|
| Other players will get called up and the world cup is the biggest stage to present yourself. |
| Players get huge contracts after world cups because they played well. |
| It simply won’t happen. |
| Major FAs like Brazil, Germany, France, England and so on could have joined forces and declined to play, that would have been an option. |
| But those are most of the time on the same level of corruption. |
| I don’t blame the players, it would have been a Kaepernick situation just in a different sport. |
Table 6: Problem 10 of PAN-25 medium training set. The transition sentence is not trivial to detect, but change of style is observed in neighboring sentences with the use of "soccer" and "football" (highlighted in orange manually) that describe the FIFA event.
| PAN-25 Hard (4 authors) |
|---|
| Soooo… according to the article, the Chinese army started scheduled training drills in the area this morning, and all of a sudden, fishermen saw a ufo? |
| The US just confirmed the others were balloons too. |
| Schumer said it on an interview on ABC this morning. |
| They haven’t blamed them on China though. |
| Not officially. |
| But he spun the announcements of the balloons into talking about China’s balloon spy program. |
Table 7: Problem 10 of PAN-25 hard training set. The topical difference is not obvious, however transition words such as "Ah", "Yeah" that indicate the response to the previous users is still presented in the transition sentence.
| PAN-26 Easy (3 authors, 5 transitions) |
|---|
| “Y/N Look out” You quickly turned your head only to see one of the Devaronians aim their blaster towards you. |
| Instead of answering, you glared daggers at him, which caused him to snicker as he made his way towards you. |
| A quick glance at your fuel reserves told you that would have ended in disaster, so, with a fake smile plastered on your face and a bowed head for anyone in uniform, you quickly explained that you were on a tour of First Order allied planets as a gift from your parents for your eighteenth birthday, and that you had been cruising around before realizing you would need to land as soon as possible to refuel. |
| “Y/N, I… I haven’t been completely honest with you, and I need to tell you this before…” “W-what do you mean, Poe? |
| “Poe—“ “Please, Y/N, let me finish or I will never get this out,” You stayed silent and Poe continued. |
| “You’re going to be a fun one to break.” As he finally released you, your gaze went straight down to Poe who continued to lay motionless on the ground. |
Table 8: PAN-26 Easy train set problem 10. Due to the nature of consistency in stroy telling, the change of topic is different than PAN-25 without transition words.
| PAN-26 Medium (20/118 lines) |
|---|
| Steve goes to Pepper for help because even though he’s only just met her and he still doesn’t know her very well, he can tell that she’s a good person with a kind heart and that she won’t say no to any reasonable request. |
| He also knows that she is the sort of person who can get things done and that she will do the things that need to be done with a minimum of fuss and bother. |
| They meet for coffee in Bryant Park on a sunny day in March and Pepper listens quietly as Steve outlines what he wants and why. |
| Pepper doesn’t blink an eye, just whips her StarkPad out of her purse and starts tapping away as they brainstorm ideas. |
| Bucky is the most alive person Steve has ever known and the idea of putting his name on a grave is unimaginably wrong. |
| He immediately vetoes her suggestion of a memorial stone in Arlinon because he just cannot bear the thought of a tomb of any type. |
Table 9: Sub-document of PAN-26 Medium train set problem 10. Due to the nature of consistency in stroy telling, the change of topic is less obvious than PAN-25. But with careful reading we can notice the location changes from park (city) to village and mountain (remote) area at the adjacent transition sentences.
| PAN-26 Hard (2 authors, 1 transition) |
|---|
| Your tears flowed freely after that and you leaned forward to press your lips to his forehead, clinging lightly to him but avoiding the injuries from the battle. |
| You sniffed, not even realizing until then that you had begun to cry, as he tried to comfort you, tears of grief and guilt clinging to his lower lashes. |
| “You nearly died Thorin,” you whispered, turning your palms to cup his bearded jaw, “You nearly died and I almost didn’t get to tell you…”. |
| “Well now Thorin, what’s going on because I can tell that it’s not anything to do with an illness.” Oin was very matter of fact when it came to these things. |
| “How did you know?” Thorin wasn’t startled by his friend and general, the dwarf did live in the same wing as the king, in fact most of the company did. |
| Thorin looked over at his general, his eyebrow raised in question, the gruff dwarf turned red and nodded. |
Table 10: Sub-document of PAN-26 Hard train set problem 11. Transition is harder to observe with the fluency of the story.
More in this topic
Stylometry
Stylometry measures patterns of style to compare authors, characterize texts, and study how writing is produced or altered.
See all 35 papers