OTel: Open Telco AI Datasets, Benchmarks, and Models

Farbod Tavakkoli; Gregory Diamos; Kenneth Church; David Kanter; Mark Austin; Imtiaz Karim; Mirza Masfiqur Rahman; Merouane Abdelkader Debbah

cs.AIOct 6, 2026

OTel: Open Telco AI Datasets, Benchmarks, and Models

Authors: Farbod Tavakkoli, Gregory Diamos, Kenneth Church, David Kanter, Mark Austin, Imtiaz Karim, Mirza Masfiqur Rahman, Merouane Abdelkader Debbah, +10 more

Organizations: AT&T Chief Data Office · RelationalAI · Northeastern University · MLCommons · The University of Texas at Dallas · Purdue University · Khalifa University · University of Leeds · Yale University · Mantis NLP · GSMA · Essential AI

Abstract

We present Open Telco (OTel), an open telecom AI resource that releases derived telecom datasets for retrieval, reranking, instruction tuning, and safety/abstention, together with 30 full-parameter post-trained baselines spanning 10 embedding models, 3 rerankers, and 17 language models. The community has already engaged substantially with the resource: as of May 3, 2026, the released models have been downloaded over 16 million times and the project has received 157+ pieces of media coverage worldwide. Building on prior open telecom datasets and benchmarks, OTel provides documented telecom data sources, held-out evaluation partitions, trained embedding models, rerankers, context-grounded LLMs, and safety/abstention data in one unified resource. Each baseline starts from an open-weight model and is post-trained on OTel-derived data using an open training recipe, then evaluated on held-out OTel evaluation partitions. OTel post-training improves performance across all three model families: embedding retrieval reaches 93.1% NDCG@10, reranking reaches 0.947 MRR@10, and language-model correctness reaches 87.8%. We release OTel as a reproducible starting point and invite the community to expand the data, improve embedding and reranking models, and build stronger context-grounded telecom LLMs.

Figures & tables

	Prior Work			This Work
Data sources & tasks	Tele-Data	TEmbed	GSMA Leaderboard	OTel
Telecom scope	Telecom QA	Telecom data	Benchmark tasks	Multi-domain telecom data
Retrieval chunks	$\times$	$\times$	Eval-only	$\checkmark$ explicit chunks
Reranker labels	$\times$	$\times$	Eval-only	$\checkmark$ relevance labels
LLM instruction data	$\checkmark$ QA-oriented	$\checkmark$ QA-oriented	Eval-only	$\checkmark$ grounded prompts
Safety	$\times$	$\times$	$\times$	$\checkmark$ abstention data

Table 1: Data sources and tasks covered by prior open telecom resources and OTel. Retrieval chunks indicate explicit positive/negative retrieval chunks. The comparison shows how OTel builds on prior work by covering the retrieval–reranking–generation workflow and safety/abstention data; it is not intended to diminish earlier datasets or leaderboards.

Contributor	Source domain	Samples
Yale University	arXiv telecom papers, 3GPP standards, telecom Wikipedia articles, and telecom-related Common Crawl pages	681,172
NetoAI	RFC series	100,751
GSMA	PRDs, Discover portal material, and mixed telecom documents	158,006
Khalifa University	Industry whitepapers	62,000
University of Leeds	O-RAN specifications across working groups 1–2, 4–10	58,565
Purdue University and UT Dallas	O-RAN documents across working groups	42,000

Table 2: Contributor summary for the raw OTel corpus.

Dataset split	Raw size	Final size
Yale enrichment path	680,000	220,334
Other contributors	420,000	106,433
Total	~1,100,000	326,767

Table 3: Attrition through the OTel cleaning pipeline.

RAG role	Dataset on Hugging Face	Task	Key fields
Retrieval	OTel-Embedding	Retrieve relevant telecom passages from an anchor/query.	anchor , positive , negative_1 – negative_5
Reranking	OTel-Reranker	Score query-passage relevance for cross-encoder reranking.	sentence_0 , sentence_1 , label
Generation	OTel-LLM	Generate grounded telecom answers from retrieved context.	prompt , completion , abstention , chunk-count metadata
Safety	OTel-Safety	Abstain when appropriate.	Schema-compatible with OTel-LLM ; includes abstention and chunk-count metadata

Table 4: Released OTel derived dataset family, ordered by the retrieval-reranking-generation workflow and its safety extension.

RAG role	Dataset	Shortened real example
Retrieval	OTel-Embedding	Retrieval example: a query about F1 Measurement ID Coordination is paired with the relevant F1-log passage and five nearby hard negatives.
Reranking	OTel-Reranker	Reranking example: an O-RAN Fronthaul Gateway query-passage pair about split 7-2 to 8 is marked relevant with label =1.0.
Generation	OTel-LLM	Generation example: an MBSFN-mode question is answered from retrieved context about TDD MBSFN Information .
Safety	OTel-Safety	Safety example: an SCP trust-domain question has off-topic retrieved contexts, so the completion abstains with abstention =true.

Table 5: Compact examples drawn from released OTel rows and shortened for readability. Full rows and regeneration commands are documented in the release materials.

Family	Count	Primary dataset
Embeddings	10	OTel-Embedding
Rerankers	3	OTel-Reranker
LLMs	17	OTel-LLM

Table 6: The 30 released full-parameter post-trained OTel baselines, ordered by the RAG workflow.

OTel model	Base model	Params (B)	Without fine-tuning	With OTel fine-tuning	$\Delta$ (pp)
OTel-LLM-270M-IT	gemma-3-270m-it	0.27	22.2	30.7 $\pm$ 1.4	+8.5
OTel-LLM-0.6B-IT	Qwen3-0.6B	0.60	49.0	58.4 $\pm$ 1.1	+9.4
OTel-LLM-1B-IT	gemma-3-1b-it	1.00	48.3	56.8 $\pm$ 1.0	+8.5
OTel-LLM-1.2B-IT	LFM2.5-1.2B-Instruct	1.20	66.4	73.8 $\pm$ 0.8	+7.4
OTel-LLM-1.7B-IT	Qwen3-1.7B	1.70	52.8	60.8 $\pm$ 0.9	+8.0
OTel-LLM-3B-IT	Mistral-3-3B	3.00	56.9	63.9 $\pm$ 0.9	+7.0

Table 7: LLM-as-judge correctness (%) on held-out OTel evaluation splits. Correctness is the average score from GPT-4o mini and Claude Sonnet 3.5. OTel fine-tuning improves correctness across all language-model baselines, and larger models generally raise the upper envelope while architecture still matters.

\Delta

is the absolute percentage-point gain. Standard errors via bootstrap (

n{=}1000

OTel model	Base model	Params (B)	Without fine-tuning	With OTel fine-tuning	$\Delta$ (pp)
OTel-Embedding-22M	all-MiniLM-L6-v2	0.022	24.1	83.8 $\pm$ 0.8	+59.7
OTel-Embedding-33M	bge-small-en-v1.5	0.033	31.4	86.3 $\pm$ 0.7	+54.9
OTel-Embedding-34M	all-MiniLM-L12-v2	0.034	29.2	84.6 $\pm$ 0.8	+55.4
OTel-Embedding-109M	all-mpnet-base-v2	0.109	38.5	87.2 $\pm$ 0.7	+48.7
OTel-Embedding-300M	Gemma3-Embedding-300M	0.300	72.3	90.4 $\pm$ 0.6	+18.1
OTel-Embedding-335M	bge-large-en-v1.5	0.335	51.7	89.2 $\pm$ 0.6	+37.5

Table 8: NDCG@10 (%) on held-out OTel evaluation splits. OTel fine-tuning improves telecom retrieval quality across embedding sizes.

\Delta

is the absolute NDCG@10 percentage-point gain. Standard errors via bootstrap (

n{=}1000

OTel model	Params (B)	Without fine-tuning	With OTel fine-tuning	$\Delta$
OTel-Reranker-0.6B	0.60	0.346	0.938 $\pm$ 0.007	+0.592
OTel-Reranker-4B	4.00	0.407	0.943 $\pm$ 0.006	+0.536
OTel-Reranker-8B	8.00	0.417	0.947 $\pm$ 0.005	+0.530

Table 9: MRR@10 on held-out OTel evaluation splits. OTel fine-tuning improves reranking quality for all three reranker baselines.

\Delta

is the absolute MRR@10 gain. Standard errors via bootstrap (

n{=}1000

Panel of Figure 1 — Figure 1: OTel fine-tuning improves language-model correctness and embedding retrieval quality; larger LLMs generally raise the correctness upper envelope.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

OTel model	Parameters	Base model
OTel-LLM-270M-IT	270M	gemma-3-270m-it
OTel-LLM-0.6B-IT	0.6B	Qwen3-0.6B
OTel-LLM-1B-IT	1B	gemma-3-1b-it
OTel-LLM-1.2B-IT	1.2B	LFM2.5-1.2B-Instruct
OTel-LLM-1.7B-IT	1.7B	Qwen3-1.7B
OTel-LLM-3B-IT	3B	Mistral-3-3B

Appendix

Table 10: Language-model subset of the 30 released full-parameter post-trained OTel baselines.

OTel model	Parameters	Base model
OTel-Embedding-22M	22M	all-MiniLM-L6-v2
OTel-Embedding-33M	33M	BAAI/bge-small-en-v1.5
OTel-Embedding-34M	34M	all-MiniLM-L12-v2
OTel-Embedding-109M	109M	all-mpnet-base-v2
OTel-Embedding-300M	300M	Gemma3-Embedding-300M
OTel-Embedding-335M	335M	BAAI/bge-large-en-v1.5

Appendix

Table 11: Embedding subset of the 30 released full-parameter post-trained OTel baselines.

OTel model	Parameters	Base model
OTel-Reranker-0.6B	0.6B	Qwen3-0.6B
OTel-Reranker-4B	4B	Qwen3-4B
OTel-Reranker-8B	8B	Qwen3-8B

Appendix

Table 12: Reranker subset of the 30 released full-parameter post-trained OTel baselines.

Model type	Dataset
LLM instruction tuning	OTel-LLM
LLM safety tuning	OTel-Safety
Embedding	OTel-Embedding
Reranker	OTel-Reranker

Appendix

Table 13: Dataset mapping for the OTel training releases.

Parameter	Value
Optimizer	AdamW (8-bit, bitsandbytes)
Learning-rate schedule	Cosine decay with warmup
Weight decay	0.01
Warmup steps	100
Per-device batch size	8–128
Gradient accumulation	4–32

Appendix

Table 14: Shared optimization template for the full-parameter post-trained OTel baselines.

Parameter	Value
Numerical precision	BF16
Attention implementation	Flash Attention 2
Gradient checkpointing	Enabled
Distributed training	Fully Sharded Data Parallel

Appendix

Table 15: Precision and distributed training configuration.

Hardware	Count
AMD MI300X	32
AMD MI325X	64
AMD MI355X	32
NVIDIA A100	8
NVIDIA H100	32

Appendix

Table 16: GPU pools used across OTel training runs.

Model type	Train	Eval	Seed
LLM	90%	10%	42
Embedding	90%	10%	42
Reranker	95%	5%	42

Appendix

Table 17: Held-out split policy used in the current OTel evaluation setup.

Logging/configuration item	Value
Evaluation strategy	Per epoch
Save strategy	Per epoch
Logging interval	Every 50 steps
Experiment tracking	TensorBoard
LLM judge models	GPT-4o mini and Claude Sonnet 3.5

Appendix

Table 18: Evaluation cadence and experiment logging defaults.

Parameter	Value
Loss masking	Completion tokens only
Label smoothing	None
Padding side	Right

Appendix

Table 19: Shared loss-side configuration.

Figure 2: Auxiliary GSMA Open Telco AI Leaderboard comparison for the non-abstention OTel-LLM-8.3B-QnA variant.

Asset class	Release unit	Documentation	Metadata
Dataset	OTel-LLM	Dataset card	Croissant + RAI
Dataset	OTel-Safety	Dataset card	Croissant + RAI
Dataset	OTel-Embedding	Dataset card	Croissant + RAI
Dataset	OTel-Reranker	Dataset card	Croissant + RAI
Model family	30 released OTel baselines	Model cards	Reproduction instructions
Code	Training/evaluation scripts	README + commands	Environment specification

Appendix

Table 20: Release artifacts and their accompanying documentation.

Source class	Sources	Availability
Standards body	3GPP specifications	Public
Consortium documents	GSMA PRDs, Discover portal	Public
O-RAN documentation	O-RAN Alliance specifications	Public
Open technical corpora	IETF RFC series	Public
Academic papers	arXiv telecom papers	Public (CC-BY)
Web-derived	Common Crawl telecom pages	Public

Appendix

Table 21: Source provenance. OTel releases only derived QA pairs, under the Apache-2.0 license.

Explore similar work

TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks
Sep 21, 2026Bohao Wang, Chenwei Wu, Hang Zou +7Large Reasoning Models Large Language Models(Llms
College of Information Science and Electronic Engineering, Zhejiang University, 310027 Hangzhou, China · Research Institute for Digital Future, Khalifa University, P O Box 127788, Abu Dhabi, UAE · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109-2122, USA +1
TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?
May 18, 2026Jieting Xiao, Yun Lin, Huizhen Qiu +10Mobile Networks
ZTE Corporation China
TeleEmbedBench: A Multi-Corpus Embedding Benchmark for RAG in Telecommunications
Apr 20, 2026Pranshav Gajjar, Vijay K ShahMobile Networks Embedder
NextG Wireless Lab, North Carolina State University, Raleigh, USA

Sep 21, 2026, cs.CLJ/K move · Enter open · S save

TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks

Bohao Wang, Chenwei Wu, Hang Zou, Yu Tian+6

College of Information Science and Electronic Engineering, Zhejiang University, 310027 Hangzhou, China · Research Institute for Digital Future, Khalifa University, P O Box 127788, Abu Dhabi, UAE · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109-2122, USA+1

Large Reasoning Models Large Language Models(Llms Chain-of-Thought Reasoning Post-Training

Open paper AskPDF arXiv

Large language models (LLMs) offer great potential to automate a broad range of telecom engineering tasks by reasoning over standards, network configurations, mathematical models, source code, and operational logs. However, existing telecom LLMs struggle to reliably reason across these diverse tasks and data types. General-purpose LLMs often lack reliable grounding in telecom-specific knowledge, while telecom-specialized models are typically developed for narrower task families and exhibit limited multi-task performance. To fill this gap, we introduce TelecomGPT-R1, a family of open source unified telecom reasoning models structured around four complementary axes: protocol, knowledge, modeling, and fault. We first develop an axis-aware data generation framework that refines coarse public telecom artifacts into verified question-answer pairs and high quality chain-of-thought (CoT) reasoning trajectories, yielding a training corpus containing 104,880 examples. Building on this corpus, supervised fine-tuning (SFT) instills telecom knowledge and evidence-grounded reasoning patterns to overcome the cold start barrier for reinforcement learning (RL). We then apply dynamic sampling policy optimization (DAPO) with task-routed rubric rewards to keep RL updates informative and stable across heterogeneous telecom reasoning tasks. These rewards decompose axis-specific CoT traces into verifiable reasoning units and combine grounded dense process credit with outcome correctness, allowing RL to learn generalizable problem solving behaviors from verifiable telecom evidence. We release the TelecomGPT-R1 models and a reproducible training recipe to support further community development. Evaluations on seven benchmarks of the GSMA Open Telco Leaderboard show that the open-source TelecomGPT-R1-27B achieves an 89.64% mean score, outperforming leading proprietary models, including GPT-5, Claude, and Gemini.

Abstract

Explore similar work

TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks

TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?

TeleEmbedBench: A Multi-Corpus Embedding Benchmark for RAG in Telecommunications