cs.CVAug 21, 2026
SaveMigrationNarrate: A Dataset for Detection of Migration Narratives in YouTube Videos
Organizations: Department of Computer Science, University of Sheffield, UK
Abstract
Narratives are central to how social communication is framed, making their detection critical for understanding and analysing public discourse. Prior work has explored narrative detection and extraction across diverse domains; however, migration narratives remain significantly understudied, primarily due to the absence of dedicated annotated datasets. Furthermore, public communication has recently shifted towards video-centric platforms, where narratives are conveyed through multimodal signals and consumed at scale. Despite this shift, narratives in videos remain largely unexplored. To bridge these gaps, we introduce MigrationNarrate, the first multimodal dataset for detection of migration narratives in the UK, consisting of 1,115 YouTube video transcripts annotated using a two-level taxonomy of 12 migration super-narratives and 53 narrative labels. This paper details the dataset design, collection, and annotations; together with benchmark results using a combination of pre-trained encoder models and both open- and closed-source Large Language Models. Finally, a thorough error analysis offers insights for future work.
Figures & tables
Figure 1: Example from MigrationNarrate dataset.
| Dataset | Source | Topic | Lang | # Narratives | # Instances |
|---|---|---|---|---|---|
| CARDS Coan et al. (2021) | Articles | Climate Change | En | 5/27 | 28,945 |
| Climate Piskorski et al. (2022) | Articles | Climate Change | En | 5/27 | 1,118 |
| COVID-19 Kotseva et al. (2023) | Articles | COVID-19 | En | 12/51 | 58,625 |
| COVID-19-Ger Heinrich et al. (2024) | Telegram | COVID-19 | De | 14 | 1,099 |
| DIPROMATS Fraile-Hernández et al. (2024) | Narratives around COVID-19 by Authorities | En/Es | 4/24 | 1,272,272 | |
| TweetIntent@Crisis Ai et al. (2024) | Ukraine–Russia War | En | 2 | 3,691 |
Table 1: Comparison of MigrationNarrate with existing public datasets for detecting narratives. # Narratives is presented as # Super-narratives/# Narratives for datasets with two-level annotation scheme.
| Immigration burdens the economy & welfare narrative | 190 |
| Immigrants do not contribute to the economy | 25 |
| Immigrants take our jobs | 10 |
| Many asylum seekers are actually economic migrants | 21 |
| Natives loose to immigrants/Immigrants receive better benefits than nationals | 32 |
| Receiving immigrants is too expensive/immigration burdens tax payers | 32 |
| Immigrants abuse the welfare system | 16 |
Table 2: MigrationNarrate super-narratives (in bold) and narratives statistics.
| Model | Setting | Super-narratives | Narratives | ||||
|---|---|---|---|---|---|---|---|
| Macro-F1 | Macro-Prec | Macro-Recall | Macro-F1 | Macro-Prec | Macro-Recall | ||
| Roberta-large | fine-tuned | 0.446 | 0.445 | 0.458 | 0.212 | 0.223 | 0.234 |
| Llama-3.1-8B-Instruct | zero-shot | 0.226 | 0.307 | 0.224 | 0.130 | 0.203 | 0.132 |
| few-shot | 0.115 | 0.296 | 0.132 | 0.028 | 0.055 | 0.041 | |
| fine-tuned | 0.525 | 0.543 | 0.525 | 0.304 | 0.309 | 0.340 | |
| Qwen2.5-7B-Instruct | zero-shot | 0.246 | 0.315 | 0.272 | 0.107 | 0.137 | 0.114 |
Table 3: Performance of the models on the test set . The best score for each evaluation measure is shown in bold, and the second-best score is underlined.
| Error Type / Explanation | Sentences from the video transcript | Gold Label | Model Predictions |
|---|---|---|---|
| Surface-Narrative Misalignment (49.7%). The model predicts a narrative aligned with salient topical cues rather than the transcript’s dominant narrative, failing to identify the speaker’s primary argumentative focus. | “On Saturday this week, a record 973 illegal migrants entered Britain on 17 boats. That is the biggest number this year. And it’s yet more evidence, as I’ve long been arguing, for why Labour’s plan for smashing the gangs isn’t going to work. With no active deterrent, with no Rwanda-style plan, with no serious approach to dissuading people from crossing the Channel, with a massive backlog in the asylum system and a pro-immigration government, this crisis on the border is going to get worse and worse and worse. ” | Political rivals’ policy failed/ is failing/ will fail | G: There are too many immigrants coming L: Our borders are strained and/or insufficiently controlled |
| Fine-Grained Label Confusion (26.67%). The model predicts a semantically related but incorrect narrative label, failing to distinguish between closely related categories and misidentifying the dominant narrative. | “British citizens are working hard, paying taxes and yet they can’t even afford a home. But the same basically is given free to the illegal migrants who are entering into the country from the day one. I mean they are given forced-fire hotel facilities, NHS facilities, dentist facilities, refugees loans and what not. All paid by the basically hardworking taxpayer and it has not changed basically from the last successive governments. The situation is getting actually from bad to worse. And there is also basically a growing sense of frustration among the people who feels that the government is actually ignoring them. I think government really need to think about it that the hardworking people paying taxes can’t afford the facilities and they have been given free to the people who are coming from other countries without contributing anything. ” | Receiving immigrants is too expensive/ immigration burdens tax payers | G: Natives loose to immigrants / Immigrants receive better benefits than nationals L: Natives loose to immigrants / Immigrants receive better benefits than nationals |
| Multi-Narrative Overlap (12.28%). The transcript contains multiple strong narratives, making more than one label plausible depending on which is treated as dominant; the ambiguity arises from the transcript itself rather than a clear model error. | “The Daily Telegraph, one of the best in the MSM, at least had the grace to feature the brave women of Essex on its front page today. […] All Yvette Cooper and the Home Office are doing, stop polluting our country. We don’t just want illegals moved. We want them out. And what happened in Epping last night was actually quite wonderful. As the locals again defied the over-the-top police force and brought only peace, even though authorities tried to lock them in cages. […] What will stop you? What will stop you? Nothing. Nothing will stop us until, until, because it’s our children. But when you say, oh yeah, your kids don’t matter then, of course they matter. Until they’re safe, until the people that are unvetted are taken away. Once they’re vetted, come back…… They did too much to support Antifa on the first night and they didn’t protect us from them at all . I’m really hoping this is to protect us from what they anticipated was coming from the left. I’m really hoping that’s what it’s about. I can’t actually imagine there are this many police to police a load of mums and dads walking to their local office. […]” | Political rivals act against the interests of the people | G: Our borders are strained and/or insufficiently controlled L: Immigration is a threat to individual safety |
| Failure to Detect Counter-Narratives (10%). The model fails to recognise that a narrative is being rejected, or corrected, and instead classifies the transcript as expressing that narrative. | “ Since the government’s policy of sending asylum seekers to Rwanda came into effect, I’ve noticed a lot of discussion about the supposed link between illegal migration and Britain’s social housing shortage. But that link isn’t quite as straightforward as some people would like to make it out to be. It’s true that in England there are currently over 1 million households waiting for social housing, but so far in 2024 around 8,200 people have arrived illegally in Britain via small boats crossing the English Channel. They are not responsible for that enormous social housing waiting list. And in fact, if you look at the government’s own data, 90% of all social housing lead tenants, that’s the person whose name will be on the lease, are British. So it’s simply not true that all of our social housing is going to illegal migrants […]” | None | G: Immigrants are a strain on the housing market L: Immigrants are a strain on the housing market |
| Weak Narrative Signal (5%). The transcript does not clearly express a strong narrative (often labelled as None), but the model predicts a narrative label based on weak topical cues or repeated keywords. | “[…] They’re going to start using TikTok influencers from Albania, Iraq, and Egypt and Iran. […] There’s been a million pounds set aside for this advertising campaign to get these influencers to encourage people to not try to cross over into the UK. […] They’re trying to tell people, look, there’s nothing for you there… Don’t go across the channel. It’s very dangerous. You might drown. Whatever they’re saying. And even when you get here, do you really want to join a machete drug gang? […] Because, of course, when people get here, they don’t want to turn around and be like, oh, I made a terrible mistake. So they post all the pictures of themselves in the hotel rooms. Like we all do on Facebook. With the £10 notes. How great our life is going. ” | None | G: Immigrants are victims of false hope L: Immigrants are victims of false hope |
Table 4: Error types and representative examples. Bold denotes our interpretation of the gold-label rationale , and red denotes our interpretation of the model’s predicted-label rationale . Both bold and red indicate the rationale for the multi-narrative overlap . Percentages are the frequency of each error type for both Llama (L) and Gemma (G).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| YouTube Channel | YouTube Playlist |
|---|---|
| @mwukofficial | All videos |
| @ReformUKOfficial | All videos |
| @labourparty | All videos |
| @Conservatives | All videos |
| @ukhomeoffice | All videos |
| @NigelFarageOfficial | All videos |
Table 5: List of YouTube channels and corresponding playlists used for data collection.
| Immigration UK |
| UK Immigration |
| Immigration in the UK |
| Migrants UK |
| UK migrant crisis |
| UK migration crisis |
| Illegal immigration UK |
Table 6: General search phrases.
Figure 2: Annotation Framework. The annotator is shown the video transcript along with a direct link to the YouTube video. First, they are presented with the list of super-narratives only. Once a super-narrative is selected, the corresponding list of narratives is displayed. Label definitions are displayed when the annotator hovers over them.
| Dataset | Total Videos | Unique Channels | Avg. Duration (Secs) |
|---|---|---|---|
| Full corpus | 5,540 | 2,081 | 90.46 |
| Human-annotated | 1,115 | 684 | 79.73 |
Table 7: MigrationNarrate statistics.
| Model | Learning Rate | Dropout |
|---|---|---|
| RoBERTa-large | 0.3 | |
| RoBERTa-large | 0.1 | |
| Llama-3.1-8B-Instruct | 0.2 | |
| Qwen2.5-7B-Instruct | 0.1 | |
| Gemma-3-12b-it | 0.1 |
Table 8: Best hyperparameter settings selected based on Macro-F1 score on the development set.
| Video transcript | Super-Narrative | Narrative |
|---|---|---|
| PM Starmer of UK getting pressure day by day. New pressure on PM Starmer and the pressure is that one more milestone of illegals are achieved. UK is breaking records even more than US regarding illegals. 33,000 boats reached in this year. 33,000 boats mean in each boat at least 100 people may be travelling. And how many people as illegals entered? You just calculate mathematically and this is an arithmetic fashion. Illegals are being increased and Stormer feeling severe pressure from all community. Thank you. Thank you very much. | Immigration is out of control | There are too many immigrants coming |
| We’ve got a demographic crisis where the data clearly shows the number of British-born children is declining. And that’s one of the reasons why lots of people are sort of saying, oh, the globalists are saying we need more mass immigration into the UK because the population… Well, maybe if you had more British-born children, then that would be a good thing. Who believe in British values, grow up with British education and understanding and culture. And that way there’s less pressure to bring in immigration from overseas. So I think it makes perfect sense. And, you know, there are three key themes, family, community and country. And strong, successful working families create strong communities. And if you’ve got strong communities, you’ve got a strong country. And that’s who we are and that’s what we’re about. I just don’t care about the labels. We just want to do what we think is right for the country. | Immigration is part of a conspiracy | Great replacement |
| This is the situation facing not only Britain but many European cities. This is a scene from London where, for better or worse, the streets support high levels of immigrant cultures due to years of legal and illegal immigration and an unwillingness for certain cultures to integrate into traditional British culture. | Immigrants’ identity and/or culture is problematic | Certain immigrants are unwilling/incapable to integrate |
| Who are the human smugglers? In Britain, it’s whoever people you are bringing in. Like if there’s people coming in from Albania, them areas, it’ll be Albanian gang. If they’re coming from India, it’s an Indian person that controls it. Right. The normal situation is 10 grand a pop and we bring you over in a container. I was talking to these people and they were saying, do you want to get involved in this, this, that, there’s this money you made. They said, but what we really do is this, yeah? We get about 30 of them in a container, 10 grand a piece, we pop them off, take them to the sea, and they never get to the destination. So we don’t actually do any work of taking them there. And they told you they were going to do this? This is what they do, yeah. | Immigrants are victims | Immigrants are victims of human traffickers |
| Migrant care workers in the UK are falling victim to exploitation, with some earning less than £5 an hour. Recent studies have shown that visa routes for both social and domestic workers foster conditions ripe for abuse, often leading to situations akin to modern slavery. Shocking findings include workers overstaying visas due to dire work conditions and paltry wages, with one individual on an overseas domestic worker visa receiving a mere £20 for their entire stay. Many cases reveal pay below the national minimum wage, inadequate accommodations with excessive rents, and constant surveillance. The government insists on taking action against such abusive practices. To understand the full extent of these issues and ongoing efforts, visit visaverge.com for comprehensive coverage. | Immigrants are victims | Immigrants suffer from labour exploitation |
Table 9: Examples from MigrationNarrate dataset.
| Muslim immigration UK |
| Islam immigration UK |
| British culture immigration |
| British identity immigration |
| Sharia law immigration UK |
| immigration out of control UK |
| immigration numbers too high UK |
Table 10: Narrative_oriented search phrases.
More in this topic
Social Media Analysis
Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.
See all 222 papers