FLANS at SemEval-2026 Task 7: RAG with Open-Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Cultures

Liliia Bogdanova; Shiran Sun; Lifeng Han; Natalia Amat Lefort; Flor Miriam Plaza-del-Arco

cs.CLMar 2, 2026

FLANS at SemEval-2026 Task 7: RAG with Open-Sourced Smaller LLMs for Everyday Knowledge Across Diverse Languages and Cultures

Authors: Liliia Bogdanova, Shiran Sun, Lifeng Han, Natalia Amat Lefort, Flor Miriam Plaza-del-Arco

Organizations: Insilico Medicine AI Limited · University of Groningen · LIACS, Leiden University · Leiden University Medical Center

Abstract

This system paper describes our participation in the SemEval-2025 Task-7 ``Everyday Knowledge Across Diverse Languages and Cultures''. We attended two subtasks, i.e., Track 1: Short Answer Questions (SAQ), and Track 2: Multiple-Choice Questions (MCQ). The methods we used are retrieval augmented generation (RAGs) with open-sourced smaller LLMs (OS-sLLMs). To better adapt to this shared task, we created our own culturally aware knowledge base (CulKBs) by extracting Wikipedia content using keyword lists we prepared. We extracted both culturally-aware wiki-text and country-specific wiki-summary. In addition to the local CulKBs, we also have one system integrating live online search output via DuckDuckGo. Towards better privacy and sustainability, we aimed to deploy smaller LLMs (sLLMs) that are open-sourced on the Ollama platform. We share the prompts we developed using refinement techniques and report the learning curve of such prompts. The tested languages are English, Spanish, and Chinese for both tracks. Our resources and codes are shared via https://github.com/aaronlifenghan/FLANS-2026

Figures & tables

Figure 1: RAG pipeline. italic indicating variations

Language	Track 1 (SAQ)			Track 2 (MCQ)
	MP	RP-v1	RP-v2	MP	RP-v1	RP-v2
English (en)	17.14	24.29	37.14	82.86	82.86	82.86
Spanish (es)	4.17	35.00	47.50	80.83	85.00	70.00
Chinese (zh)	27.14	41.43	48.57	65.71	82.86	68.57
Average	16.15	33.57	44.40	76.46	83.57	73.81

Table 1: RAG-base Prompt ablation results by language and task.

Language	Track 1 (SAQ)			Track 2 (MCQ)
	RP-v1	RP-v2	RP-v1 (no local DB)	RP-v1	RP-v1 (no local DB)
English (en)	16.67	0.00	16.67	83.33	83.33
Spanish (es)	33.33	0.00	33.33	61.11	72.22
Chinese (zh)	33.33	8.33	66.67	91.67	83.33
Average	27.78	2.78	38.89	78.70	79.63

Table 2: RAG-web Prompt ablation results by langauge and task.

Figure 2: RAG-base Learning curves of three prompt ablation averaged over three-languages (en, es, zh).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: RAG-base pipeline using smaller LLMs favoring Gemma3.4b - keywords based KE - then land to KB

Figure 4: RAG-web pipeline using mistral:7b and deepseek-llm:67b (bigger) - Conditional adaptive RAG

Multilingual QA

Multilingual question answering studies how AI systems answer questions across languages using cross-lingual reasoning, retrieval, and culturally grounded knowledge.

See all 63 papers

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering
Sep 16, 2026Pablo Poulenard, Yannis Karmim, Valentin BarrièreRetrieval-Augmented Generation Graph Retrieval-Augmented Generation
DCC, Universidad de Chile, Santiago, Chile · École Polytechnique, Palaiseau, France · Inria, Almanach, Paris, France +2
SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures
May 4, 2026Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros +27Multilingual Language Model Evaluation LLM Evaluation
Cardiff University · KAIST · MBZUAI +13
Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG
Jun 5, 2026Sifei Meng, Dmitry IlvovskyCross-Encoder Reranking Retrieval-Augmented Generation
Faculty of Computer Science HSE University

Sep 16, 2026J/K move · Enter open · S save

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering

Pablo Poulenard, Yannis Karmim, Valentin Barrière

DCC, Universidad de Chile, Santiago, Chile · École Polytechnique, Palaiseau, France · Inria, Almanach, Paris, France+2

Retrieval-Augmented Generation Graph Retrieval-Augmented Generation Question Answering Cultural Knowledge in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) suffer from a long-tail deficit: culturally specific facts, particularly those concerning underrepresented regions such as Latin America, appear too rarely in pretraining corpora to be reliably memorized. Retrieval-Augmented Generation (RAG) addresses this by grounding generation in external text, but structured alternatives such as Knowledge Graphs (KGs) offer tighter control over what enters the context, along with potential gains in explainability and updatability. We benchmark Graph-RAG against standard RAG on LatamQA, a culturally grounded multiple-choice dataset spanning eight thematic categories. The graphs are built end-to-end from Wikipedia articles with KGGen, a recent open-domain extractor, without manual curation in our main setting. G-Retriever is competitive with RAG and reduces the error of the base LLM by 72% with a standard KG and 78% with a benchmark-aware variant, the gap to RAG narrowing further as the graph is oriented toward task-relevant content. The trained projection transfers zero-shot to Portuguese without target-language fine-tuning, indicating multilingual reach.

Abstract

Multilingual QA

Explore similar work

Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering

SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures

Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG