Building Telecom Language Models with RSOD: Domain Data, Specialist Post-Training, Memory-Conscious Distillation, and Networking Evaluation

Anil Srirangapatna Nagesh; Nitin Gupta; Anuj Taneja; Vivek Singh; Shane Eleniak; Theo Zhao; Moulidharan N; Shashank Keshetty

cs.AIOct 20, 2026

Building Telecom Language Models with RSOD: Domain Data, Specialist Post-Training, Memory-Conscious Distillation, and Networking Evaluation

Authors: Anil Srirangapatna Nagesh, Nitin Gupta, Anuj Taneja, Vivek Singh, Shane Eleniak, Theo Zhao, Moulidharan N, Shashank Keshetty

Organizations: Calix AI Labs

Abstract

Telecom language models must combine broad networking knowledge with specialist behavior across routing, broadband services, access technologies, and operational diagnostics while remaining practical to deploy as a single model. We present an end-to-end framework that builds source-grounded synthetic training data from structured networking taxonomies and expert graphs, materializes 212,559 routed training records across fourteen networking routes, trains six frozen specialist LoRA teachers over one shared continued-pretrained backbone, and consolidates their behavior into one deployable student adapter using Routed Specialist On-Policy Distillation (RSOD). In RSOD, the committed student generates each trajectory, the route-assigned specialist teacher-forces the exact native token sequence, compact teacher hidden states are reconstructed through the shared frozen output head, and only the student LoRA is updated. A controlled reconstruction validation shows mean absolute logit error of 7.26 x 10^-6 with 100% top-1 agreement between native and reconstructed teacher logits. On four RTX 6000 Ada GPUs, the completed 2,000-step run produced 16,000 trajectories with zero observed student-version or token- ID mismatches. Under the primary route-conditioned evaluation over 2,850 held-out questions, the raw Meta-Llama-3.1-8B-Instruct model scores 12.8661/25 micro (13.0172/25 macro), the combined routed six-teacher system scores 21.7641/25 micro, and the consolidated RSOD student, released as Llama-CalixCLM v1, reaches 21.7112/25 micro (21.8006/25 macro) on the same five dimension LLM-judge rubric. Under a separate neutral deployment-like prompt, the same student reaches 20.0028/25 micro (20.0778/25 macro). The consolidated student remains within 0.0529 micro-score points of the combined routed-teacher system while retaining a numerically validated and auditable train-serve execution path.

Figures & tables

Term	Meaning
RSOD	Routed Specialist On-Policy Distillation
CPT	Continued Pretraining
SFT	Supervised Fine-Tuning
KD	Knowledge Distillation
KL	Kullback–Leibler divergence
LoRA	Low-Rank Adaptation

Table 1: Abbreviations used in the paper.

Dimension	Established building block	RSOD configuration
Trajectories	Student-on-policy distillation	Route-conditioned student trajectories generated by the committed student
Teacher choice	Multi-teacher selection or aggregation	Deterministic route-to-specialist assignment; one teacher per trajectory
Token alignment	Teacher scoring of generated output	Exact native token-ID teacher forcing on the student’s sequence
Signal transport	Logits or local teacher execution	Teacher hidden-state transport with local shared LM-head logit reconstruction
Model serving	Shared multi-adapter serving	One shared teacher backbone plus version-acknowledged student adapter commits
Validation	Aggregate training metrics	Direct token-identity and serving-version audits over the run

Table 2: RSOD combines established building blocks into an audited telecom-specialist consolidation workflow.

Figure 1: End-to-end framework. Structured networking data supports one shared CPT backbone and six frozen specialist LoRAs. RSOD routes each training trajectory to one specialist, trains a single student adapter, records run provenance and audits, and produces one deployable student.

Specialist	Train records	Heldout
Dynamic Routing	45,597	570
MPLS Services	26,439	190
Broadband Services	29,360	570
Routing Fundamentals	52,083	380
CLI Diagnostics	20,990	570
Edge Access	38,090	570

Table 3: Routed training and held-out inventory used by the final six-specialist configuration.

Specialist	Route	Train	Eval.	Specialist	Route	Train	Eval.
Dynamic Routing	BGP	19,756	190	CLI Diagnostics	Linux tool calling	5,817	190
Dynamic Routing	OSPF	9,561	190	CLI Diagnostics	Linux agentic diagnosis	5,817	190
Dynamic Routing	IS-IS	16,280	190	CLI Diagnostics	Network fundamentals	9,356	190
MPLS Services	MPLS VPN	26,439	190	Edge Access	L2 switching	11,061	190
Broadband Services	BNG/subscriber mgmt.	24,735	380	Edge Access	PON/optical	8,296	190
Broadband Services	QoS	4,625	190	Edge Access	WiFi gateway	18,733	190

Table 4: Canonical route-to-specialist mapping and held-out evaluation inventory. The held-out set contains 190 questions per route except BNG/subscriber management with 380.

Figure 2: Detailed RSOD execution sequence. The deterministic route scheduler chooses one frozen specialist teacher; the committed student generates the native rollout, and the teacher scores the same token sequence by teacher forcing. Teacher and student hidden states are mapped through the shared frozen normalization/output head, forward KL is computed on generated positions, and only the student LoRA is updated. The candidate adapter is refreshed, acknowledged active, and committed before the next rollout.

Figure 3: Systems summary. (a) Mean and peak device VRAM across the four GPUs. (b) P50/P95 rank-0 host-boundary latency across major phases, including version commit/acknowledgement. (c) Trainer-side hidden-state materialization and analytical dense FP32 logit accounting.

Figure 4: Per-device telemetry for the run. (a) GPU power statistics. (b) Estimated GPU energy by device from NVML sampling. (c) GPU temperature statistics.

Metric	Value
Optimizer steps	2,000
Trajectories	16,000
KD tokens	2,926,443
Active training time	50,511 s
GPU-hours	56.1
Estimated GPU energy	4.63 kWh

Table 5: RSOD run summary. Energy is NVML-derived; VRAM is aggregate device usage.

Figure 5: Per-specialist-group workload and runtime statistics. Panels show trajectory counts, KD-token totals, mean generated length, mean teacher-signal size, teacher batch-inference latency, and mean KD tokens per trajectory.

Panel of Figure 6 — Figure 6: Optimization traces over 2,000 steps: forward-KL training loss with P10–P90 band (top), learning-rate schedule (bottom left), and mean student gradient norm (bottom right).

Language Model Post-Training

Language model post-training adapts pretrained models to follow instructions, reason, use tools, and meet task-specific or human-defined goals.

See all 190 papers

OTel: Building Domain-Specialized Telecom LLM Foundations for Intelligent Networks
Aug 15, 2026Farbod Tavakkoli, Roderic Paulk, Jorden Terrazas +9Domain Adaptation for LLMs Language Model Post-Training
AT&T Chief Data Office · GSMA · RelationalAI +4
TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks
Sep 21, 2026Bohao Wang, Chenwei Wu, Hang Zou +7LLM Grounding RL for Language Model Reasoning
College of Information Science and Electronic Engineering, Zhejiang University, 310027 Hangzhou, China · Research Institute for Digital Future, Khalifa University, P O Box 127788, Abu Dhabi, UAE · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109-2122, USA +1
OTel: Open Telco AI Datasets, Benchmarks, and Models
Oct 6, 2026Farbod Tavakkoli, Gregory Diamos, Kenneth Church +14Language Model Post-Training LLM Reranking
AT&T Chief Data Office · RelationalAI · Northeastern University +6

Aug 15, 2026J/K move · Enter open · S save

OTel: Building Domain-Specialized Telecom LLM Foundations for Intelligent Networks

Farbod Tavakkoli, Roderic Paulk, Jorden Terrazas, Kenneth Church+8

AT&T Chief Data Office · GSMA · RelationalAI+4

Domain Adaptation for LLMs Language Model Post-Training LLM Grounding LLM Reranking

Open paper AskPDF arXiv

Frontier AI models have advanced rapidly, but they still struggle with telecom-specific tasks. We present Open Telco (OTel), an open telecom AI resource with derived datasets for retrieval, reranking, instruction tuning, and safety/abstention, plus 30 full-parameter post-trained baselines across embedding, reranking, and language models. The community has already engaged substantially with the resource: as of May 3, 2026, the released models have been downloaded over 16 million times, and the project has received 157+ pieces of media coverage worldwide. Building on prior open telecom datasets and benchmarks, OTel provides documented telecom data sources, held-out evaluation partitions, trained embedding models, rerankers, context-grounded LLMs, and safety/abstention data in one unified resource. OTel post-training improves performance across all three model families: embedding retrieval reaches 93.5% NDCG@10, reranking reaches 0.952 MRR@10, and language-model correctness reaches 88.2%. We release OTel as a reproducible starting point and invite the community to expand the data, improve embedding and reranking models, and build stronger context-grounded telecom LLMs.

Abstract

Language Model Post-Training

Explore similar work

OTel: Building Domain-Specialized Telecom LLM Foundations for Intelligent Networks

TelecomGPT-R1: Unified Post-Training for Reasoning Across Heterogeneous Telecom Tasks

OTel: Open Telco AI Datasets, Benchmarks, and Models