HYVE: Hybrid Views for LLM Context Engineering over Machine Data

Jian Tan; Fan Bu; Yuqing Gao; Dev Khanolkar; Jason Mackay; Boris Sobolev; Lei Jin; Li Zhang

cs.AIApr 7, 2026

HYVE: Hybrid Views for LLM Context Engineering over Machine Data

Authors: Jian Tan, Fan Bu, Yuqing Gao, Dev Khanolkar, Jason Mackay, Boris Sobolev, Lei Jin, Li Zhang

Organizations: Cisco Systems, Inc., San Jose, California, USA

Abstract

Machine data is central to observability and diagnosis in modern computing systems, appearing in logs, metrics, telemetry traces, and configuration snapshots. When provided to large language models (LLMs), this data typically arrives as a mixture of natural language and structured payloads such as JSON or Python/AST literals. Yet LLMs remain brittle on such inputs, particularly when they are long, deeply nested, and dominated by repetitive structure. We present HYVE (HYbrid ViEw), a framework for LLM context engineering for inputs containing large machine-data payloads, inspired by database management principles. HYVE surrounds model invocation with coordinated preprocessing and postprocessing, centered on a request-scoped datastore augmented with schema information. During preprocessing, HYVE detects repetitive structure in raw inputs, materializes it in the datastore, transforms it into hybrid columnar and row-oriented views, and selectively exposes only the most relevant representation to the LLM. During postprocessing, HYVE either returns the model output directly, queries the datastore to recover omitted information, or performs a bounded additional LLM call for SQL-augmented semantic synthesis. We evaluate HYVE on diverse real-world workloads spanning knowledge QA, chart generation, anomaly detection, and multi-step network troubleshooting. Across these benchmarks, HYVE reduces token usage by 50-90% while maintaining or improving output quality. On structured generation tasks, it improves chart-generation accuracy by up to 132% and reduces latency by up to 83%. Overall, HYVE offers a practical approximation to an effectively unbounded context window for prompts dominated by large machine-data payloads.

Figures & tables

Fig. 1: HYVE architecture. The preprocessor parses input strings with embedded structured segments into hybrid tabular views; the request-scoped datastore retains the full data; and the postprocessor inspects LLM outputs and, when needed, queries the datastore to enrich the response.

Panel of Figure 2 — Fig. 2: Slot-filling tasks over 700 network telemetry samples. Top: baseline GPT-4.1 consumes ${\sim}$ 5000 tokens per request; HYVE reduces usage by 53%. Bottom: baseline GPT-4.1 fails on 30.6% of samples (scores $\leq$ 2); HYVE achieves almost perfect scores on all samples.

Panel of Figure 3 — Fig. 3: Line chart over three years of USD exchange-rate data (778 points per series). Baseline GPT-4.1 truncates output arrays to 40–120 points, collapsing detailed time series into sparse straight-line segments. HYVE recovers all 778 points through data backfilling.

Fig. 4: Layout of $\mathsf{Rep}(s)$ . If table truncation is triggered, SQL query prompt is automatically generated.

	Model	Cert-QA G	Runbook G	Line S	Bar S	Anom R	Sum G	Canvas R	RB-Text G	RB-JSON R	Hard R	TOON-QA E
Score $\uparrow$	GPT-5	4.49	4.55	0.68	0.85	3.28	3.82	4.96	4.67	4.84	4.33	0.96
	GPT-5 + HYVE	4.49	4.58	0.97	1.00	3.77	3.85	4.96	4.77	4.92	5.00	0.98
	GPT-4.1	4.36	4.26	0.41	0.99	3.22	4.43	4.94	4.88	4.78	4.04	0.47
	GPT-4.1 + HYVE	4.38	4.32	0.95	0.99	4.03	4.46	4.95	4.89	4.92	5.00	0.93
Token $\downarrow$	GPT-5	3.1M	1.1M	4.2M	1.8M	18.5M	381.3K	122.8M	71.7K	142.8K	80.5K	1.4M
	GPT-5 + HYVE	3.1M	1.1M	748.0K	251.8K	10.9M	372.4K	38.2M	56.8K	132.7K	20.3K	182.4K

TABLE I: Comparison of answer quality, token usage, and latency across benchmarks

Panel of Figure 6 — Fig. 5: Per-sample distributions on the Line chart dataset ( $n=100$ ). HYVE raises 76% of samples to perfect similarity (1.0), compared with 17% for the baseline, while reducing median latency from 47.9s to 8.9s.

Panel of Figure 7 — Fig. 6: Per-sample distributions on the Anom dataset (n=797). HYVE improves reasoning quality while reducing token consumption through the anomaly operator’s high-signal summarization.

	Variant	Bar S	Anom R	Canvas R	RB-Text G	Hard R	TOON-QA E
Score $\uparrow$	Full	0.99	4.03	4.95	4.89	5.00	0.93
	Full $-$ rank	0.99	4.03	4.95	4.56	5.00	0.91
	Full $-$ trunc	0.73	3.83	4.93	4.56	4.10	0.87
	Full $+$ TOON	0.97	4.02	4.85	4.67	5.00	0.92
	Full $-$ post	0.06	3.41	4.94	4.78	3.93	0.38
Token $\downarrow$	Full	132.5K	8.9M	35.1M	48.4K	15.1K	153.8K

TABLE II: Ablation study of HYVE with GPT-4.1

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 248 papers

Mar 30, 2026

Meta-Harness: End-to-End Optimization of Model Harnesses

The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens. On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.

Yoonho Lee, Roshen Nair, Qizheng Zhang +3

Stanford · KRAFTON · MIT

LLM Agent Harnesses Large Language Model-Guided Optimization

Ask PDF

Jul 11, 2026

Structured Thoughts For Improved Reasoning And Context Pruning

Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating <try> and <outcome> blocks: <try> captures exploratory scratch work, while <outcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into <try> blocks and prompting an LLM to summarize each step into its corresponding <outcome>. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each <try>/<outcome> pair, the <try> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85% memory / context savings with an 8.67% performance drop across mathematical tasks.

Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu +5

University of Chicago · Capital One

Efficient Language Model Reasoning Structured CoT Reasoning CoT Reasoning LLM Reasoning Structured Reasoning LLM Context Management+1

Ask PDF

Sep 1, 2026

Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives

Large language models (LLMs) augmented with external tools have demonstrated remarkable capability in solving complex real-world tasks. However, existing approaches suffer from two key challenges: brittle multi-step and multi-turn reasoning caused by incompatible tool output types and API schemas, and performance degradation under large tool catalogues. To address these, we introduce Tool Primitives, a design that replaces rigid API schema-based invocation with natural language as the interface for tool calling, where each tool is wrapped with an LLM interface that handles schema resolution and execution internally, enabling natural inter-tool communication for nested and multi-turn tool calling. Building on Tool Primitives, we host ToolFace, a centralized repository of 25,519 functions from which LLMs dynamically retrieve only the relevant tools at inference time, eliminating the need to enumerate raw API schemas in context. To orchestrate Tool Primitives and ToolFace reliably in complex settings, we further propose HEART, a Harness Engineering framework via Agent-native, Reusable Tool Primitives, comprising a Planner, Router, and Verifier that jointly support dynamic tool invocation planning, multi-step execution, and feedback-driven recovery. Experiments on five benchmarks demonstrate that HEART outperforms SFT-based models by

10\%

on average and surpasses GPT-5.4, Claude-4.6-Sonnet, and Gemini-3.1-Pro by

6\%

on average while reducing API cost by up to

85\%

. On 50 real-world tasks, HEART achieves

84\%

task completion,

3.8\times

the average of three frontier commercial models (

22\%

Haibo Jin, Suijin Wang, Xucheng Yu +2

School of Information Sciences University of Illinois at Urbana-Champaign Champaign, IL 61820 · Independent Researcher Starc Institute

Tool-Augmented Language Model Agents LLM Agent Orchestration LLM Tool Use Tool-Using Language Agents LLM Agent Harnesses

Ask PDF

Abstract

LLM Context Management

Explore similar work

Meta-Harness: End-to-End Optimization of Model Harnesses

Structured Thoughts For Improved Reasoning And Context Pruning

Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives