How Far Can Lightweight LLMs Go in Intent-to-SLO Translation?
Organizations: School of Computing, Queen’s University, Ontario, Canada
Abstract
Intent-based networking (IBN) promises QoS provisioning in which the network derives the configuration from operator-stated goals. Its first step, translating a natural-language intent into enforceable service-level objectives (SLOs) grounded in a live snapshot of the network state, increasingly relies on large language models (LLMs). These large models add cost and energy to the management loop and typically require multi-GPU, cloud-hosted infrastructure. Can lightweight open LLMs take over this step dependably? We answer this question with a five-dimensional evaluation of ten LLMs with <10B parameters from five families, under four prompting setups, on 716 placement intents. We evaluate translation performance, intent validation, hallucination, energy impact, and robustness to diverse phrasing. The best configurations translate intents with a strict F1 of 0.943 and reject up to 191 of 195 invalid intents. With few-shot reasoning, the strongest models hallucinate unsupported QoS requirements in only about one response in a hundred, whereas the zero-shot setup struggles. A translated intent costs 0.044-0.148 Wh of GPU energy without reasoning, and added energy cost for reasoning pays off for the weaker models. Furthermore, we show that realistic rephrasing lowers the rejection of invalid intents in 15 of 16 configurations for the best-performing models, leaving robust intent validation as a key open challenge for reliable IBN. Our code and data are available: https://github.com/ggck43/intent2slo.
Figures & tables
| Orig. | For svc, please run it on the edge side, make sure the target node has at least 2 GB of memory, and pick the node with the highest storage capacity. |
|---|---|
| C1 | We are retiring a couple of old machines at the end of the month, and the recommender is waiting for a new home. Please run svc on the edge side with a node that offers at least 2 GB of memory, and choose the one with the highest storage capacity. I’ll pass the result to the migration team tomorrow, so thank you for a quick reply. |
| C2 | The recommender has moved between hosts twice since spring and the team wants it settled. Choose the node with the highest storage capacity for svc, make sure it has 2 GB of memory or more, and keep it on the edge side. |
| C3 | Service: svc - Layr: edge side pls - Memory: node need at least 2 GB of mem - Storage: pick the node what have the highest storge capacity Migration is soon so asap if you can. Thx, i really apreciate it. |
| zs | fs | zs-cot | fs-cot | Mean | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Family | Size | EM | EM | EM | EM | |||||
| Qwen3.5 | 4B | 0.918 | 0.820 | 0.910 | 0.774 | 0.892 | 0.772 | 0.929 | 0.845 | 0.912 |
| 9B | 0.921 | 0.835 | 0.922 | 0.802 | 0.942 | 0.879 | 0.930 | 0.823 | 0.929 | |
| Gemma 4 | E2B | 0.591 | 0.397 | 0.602 | 0.399 | 0.850 | 0.695 | 0.886 | 0.747 | 0.732 |
| E4B | 0.726 | 0.403 | 0.872 | 0.733 | 0.883 | 0.754 | 0.943 | 0.879 | 0.856 | |
| Ministral 3 | 3B | 0.408 | 0.159 | 0.714 | 0.493 | 0.479 | 0.236 | 0.847 | 0.702 | 0.612 |
| Intent (abridged) | Ground-truth SLOs | Model (setup): hallucinated SLO | Why it is a hallucination |
|---|---|---|---|
| “Deploy svc … at least 50 GB of storage capacity and pick the node with the lowest current memory utilization” (valid) | storage cap. 50 GB; memory util. [lowest] 34.27 % | Qwen3.5-9B ( zs ): memory util. [lowest] 18.0 % | 18.0 is the storage utilization of the correct node: right node, wrong metric |
| “I would like to place svc and keep CPU utilization below 40 %” (valid) | CPU util. 40 % | Qwen3.5-9B ( fs-cot ): placement = edge | a placement layer the intent does not name |
Intent-Based Networking
Intent-Based Networking translates high-level goals into validated network policies and uses feedback to manage services, resources, and performance.
See all 24 papers