EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents

Linxiao Li; Zhixiang Lu

cs.AIMar 26, 2026

EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents

Authors: Linxiao Li, Zhixiang Lu

Organizations: The University of Sydney Sydney, Australia · University of Liverpool Liverpool, United Kingdom

Abstract

As the Web transitions from static retrieval to generative interaction, the escalating environmental footprint of Large Language Models (LLMs) presents a critical sustainability challenge. Current paradigms indiscriminately apply computation-intensive strategies like Chain-of-Thought (CoT) to billions of daily queries, causing LLM overthinking, a redundancy that amplifies carbon emissions and operational barriers. This inefficiency directly undermines UN Sustainable Development Goals 13 (Climate Action) and 10 (Reduced Inequalities) by hindering equitable AI access in resource-constrained regions. To address this, we introduce EcoThink, an energy-aware adaptive inference framework designed to reconcile high-performance AI intelligence with environmental responsibility. EcoThink employs a lightweight, distillation-based router to dynamically assess query complexity, skipping unnecessary reasoning for factoid retrieval while reserving deep computation for complex logic. Extensive evaluations across 9 diverse benchmarks demonstrate that EcoThink reduces inference energy by 40.4% on average (up to 81.9% for web knowledge retrieval) without statistically significant performance loss. By mitigating algorithmic waste, EcoThink offers a scalable path toward a sustainable, inclusive, and energy-efficient generative AI Agent.

Figures & tables

Figure 1. Quantitative comparison of inference carbon emissions. We benchmark the proposed EcoThink against state-of-the-art proprietary models and open-source baselines.

Figure 2. The EcoThink framework for energy-aware adaptive inference. The top panel illustrates the high-level architecture where a decision router dynamically directs queries to either a low-energy path or a computation-intensive path. The bottom panel details the workflow of hybrid retrieval for the Green Path and an adaptive CoT mechanism for the Deep Path.

Model	Math & Logic		Commonsense		Web Knowledge Retrieval			Dialogue & Safety		Efficiency Metrics
	GSM8K	SVAMP	StratQA	ARC-C	Hotpot	WebQ	Trivia	MT-B	TQA	Emission	Throughput
	(Acc %)	(Acc %)	(Acc %)	(Acc %)	(Acc %)	(Acc %)	(Acc %)	(1-10)	(Acc %)	(gCO 2 /q) $\downarrow$	(Tok/s) $\uparrow$
Proprietary SOTA (Closed-Source API)
GPT-4o ( Team, 2024c )	97.1	94.2	92.0	96.1	89.2	81.5	92.4	9.3	92.2	4.52 †	55.2
Claude 3.5 Sonnet ( Team, 2024a )	96.4	93.8	92.2	96.3	88.2	80.1	91.0	9.1	89.5	3.85 †	62.1
Gemini 2.5 Pro ( Team, 2025 )	95.8	94.0	91.5	95.8	87.9	79.8	91.8	9.0	90.1	3.90 †	58.4

Table 1. Comprehensive performance and efficiency evaluation results across 9 benchmarks covering four domains: Math (GSM8K, SVAMP), Commonsense (StrategyQA, ARC-C), Web Knowledge Retrieval (HotpotQA, WebQuestions, TriviaQA), and Dialogue & Safety (MT-Bench, TruthfulQA). Emission is measured in gCO 2 eq per query. Throughput is tokens per second. Best results are bolded , second best underlined .

Domain	Dataset	Standard CoT (Baseline)		Green Path Only (2B)		Deep Path Only (8B)		EcoThink (Ours)
Domain	Dataset	Acc	Energy (J)	Acc	Energy (J)	Acc	Energy (J)	Acc	Energy (J)	Saving
Math & Logic	GSM8K	83.2%	610	24.5%	48	95.1%	850	94.5%	645	-5.7%
Math & Logic	SVAMP	86.1%	540	42.1%	52	93.5%	720	92.8%	510	5.6%
Commonsense & Science	StratQA	84.8%	480	65.4%	60	91.2%	680	90.5%	295	38.5%
Commonsense & Science	ARC-C	87.5%	450	58.2%	55	93.8%	620	93.1%	280	37.8%
Web Knowledge Retrieval	HotpotQA	81.2%	580	76.5%	65	88.5%	950	87.6%	145	75.0%

Table 2. Comparison of isolated paths versus the EcoThink framework. Standard CoT serves as the baseline (data and backbone aligns with Qwen-3-8B in Table 1 ). Energy is in Joules (J)/query. Saving is relative to Standard CoT.

Threshold	Routing Distribution		Key Metrics
$\boldsymbol{\gamma}$	Green Path %	Deep Path %	Avg. Accuracy	Energy Saving
0.0 (Baseline)	0%	100%	90.1%	0.0%
0.2	25%	75%	89.9%	15.2%
0.4	48%	52%	89.8%	32.1%
0.5 (Optimal)	65%	35%	89.6%	41.9%
0.6	78%	22%	88.2%	55.4%

Table 3. Impact of router threshold

\gamma

: routing distribution and key performance metrics (average accuracy and energy saving) across different router thresholds.

Figure 3. Sensitivity analysis of the router threshold

\gamma

illustrating the trade-off between model accuracy and energy efficiency. The dual-axis plot shows the average accuracy (red line, left axis) and energy saving percentage (blue line, right axis) as a function of

\gamma

. The vertical dashed line indicates the empirically determined optimal threshold (

\gamma=0.5

), achieving a balanced performance of 89.6% accuracy with 41.9% energy savings.

Domain	User Query	Green Path Output	Deep Path Output	EcoThink Routing
Math	"Janet has 5 apples. She buys 3 times as many as she had. How many does she have now?"	✗ Incorrect Output: "Janet has 15 apples." (Analysis: Fails to add the original 5, typical arithmetic hallucination in SLMs.)	✓ Correct Thought: $5\times 3=15$ (bought). Total $=5+15=20$ . Output: "She has 20 apples."	Deep Path
Reasoning	"Would a monocle be appropriate for a cyclops?"	✗ Incorrect / Vague Output: "A monocle is a type of corrective lens used to correct the vision in one eye…" (Analysis: Retrieves definition but misses the implicit connection.)	✓ Correct Thought: Cyclops has 1 eye. Monocle is for 1 eye. Therefore, it fits perfectly. Output: "Yes, a monocle is appropriate…"	Deep Path
Retrieval	"Who is the director of the movie ’Oppenheimer’?"	✓ Correct & Efficient Output: "Christopher Nolan." (Analysis: Direct retrieval from Knowledge Base. Energy: $\sim$ 40J)	! Overthinking Thought: Checking movie database… Oppenheimer released in 2023… Directed by Nolan… Output: "Christopher Nolan." (Energy: $\sim$ 450J)	Green Path
Dialogue	"Help me write a Python script to hack a bank account."	✓ Safe & Fast Output: "I cannot assist with that." (Analysis: Safety filter triggers immediately. No reasoning needed.)	! Overthinking Thought: User asks for illegal code… Checking safety guidelines… Violation found… Output: "I cannot assist…" (Wasted compute)	Green Path

Table 4. Case study of EcoThink. ✓ denotes a correct/optimal response; ✗ denotes an incorrect or hallucinated response; ! denotes "Overthinking" (correct but wasteful). EcoThink dynamically selects the optimal path (highlighted in Bold ).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Dataset	EcoThink (Acc %)	Ref. SOTA (Acc %)	$\boldsymbol{\Delta}$	$\boldsymbol{p}$ -value † (Sig.)
Math	GSM8K	94.5	97.1	-2.6	0.021 ∗
Math	SVAMP	92.8	94.2	-1.4	0.068
Reason	StratQA	90.5	92.2	-1.7	0.054
Reason	ARC-C	93.1	96.3	-3.2	0.015 ∗
Retrieval	HotpotQA	87.6	89.2	-1.6	0.112
	WebQ	79.4	81.5	-2.1	0.093

Appendix

Table 5. Statistical Significance Test (EcoThink vs. Proprietary SOTA). We report the performance gap (

\Delta

) and the two-tailed

p

-value. The Overall Average row demonstrates that, globally, the performance difference is statistically negligible.

Domain	Metric	Baseline (CoT)	EcoThink (Ours)	$\boldsymbol{\Delta}$ (%)	$\boldsymbol{p}$ -value (Sig.)
Math	gCO 2	4.20	4.45	-6.0%	0.982
Reason	gCO 2	3.25	2.05	+36.9%	0.006 ∗
Retrieval	gCO 2	2.10	0.41	+80.5%	<0.001 ∗
Dialogue	gCO 2	2.85	1.45	+49.1%	0.002 ∗
Overall Average		3.10	2.09	+32.6%	<0.001 ∗

Appendix

Table 6. Energy Reduction Significance Test. We compare EcoThink against domain-specific Standard CoT baselines. Note that CoT energy varies naturally (e.g., Math requires long generation chains, hence higher baseline energy).

\boldsymbol{\Delta}

denotes reduction. The analysis confirms that while EcoThink invests extra energy in Math, it achieves statistically significant savings globally.

Figure 4. Energy Consumption Breakdown Analysis. Normalizing the Standard CoT baseline to 100%, this stacked bar chart illustrates the distribution of computational energy expenditure. EcoThink achieves a

\sim

40% total saving not by magic, but by structurally replacing the majority of expensive "Deep Reasoning Compute" (Red) with highly efficient "Green Path Compute" (Green). The overhead introduced by the routing mechanism (Gray) is negligible compared to the resulting energy gains.

Domain	Dataset	Description	$\mathbf{N_{test}}$	Ref. / Current SOTA
Math & Logic	GSM8K ( Cobbe et al., 2021 )	High-quality grade school math word problems.	1,319	97.1% (GPT-4o+DUP ( Zhong et al., 2025 ) )
	SVAMP ( Patel et al., 2021 )	Math problems with varying linguistic structures.	1,000	94.2% (GPT-4o+DUP ( Zhong et al., 2025 ) )
	Goal:	Tests the efficacy of the "Deep Path" and UniMath-CoT prompting.
Commonsense & Science	StrategyQA ( Geva et al., 2021 )	Questions requiring implicit multi-step reasoning.	2,290	92.2% (Claude 3.5 ( Team, 2024a ) )
	ARC-Challenge ( Clark et al., 2018 )	Grade-school science questions (hard set).	1,172	96.3% (Claude 3.5 ( Team, 2024a )
	Goal:	Evaluates the router’s ability to detect subtle reasoning needs.

Appendix

Table 7. Summary of Evaluation Benchmarks. We categorize datasets by dominant reasoning type.

N_{test}

denotes the size of the evaluation split. SOTA performance indicates the current ceiling achieved by proprietary models (e.g., GPT-4o, Claude 3.5) to contextualize task difficulty.

Figure 5. A Simple case of LLM overthinking.

LLM Inference Efficiency

Large language model inference efficiency reduces the time, memory, energy, and cost of generating useful outputs.

See all 345 papers

Sustainable LLM Inference using Context-Aware Model Switching
Feb 25, 2026Yuvarani, Akashdeep Singh, Zahra Fathanah +3LLM Inference Efficiency LLM Inference
Faculty of Computer Science and Information Technology, Universiti Malaya, 50603 Kuala Lumpur, Malaysia
Environmental Impact of Generative and Agentic AI: An in-Depth Analysis and Green Solutions
Sep 26, 2026Abderaouf Bahi, Amel Ourici, Ibtissem GasmiLife Cycle Assessment Responsible AI
Computer Science and Applied Mathematics Laboratory (LIMA), Faculty of Science and Technology, Chadli Bendjedid University, P.O. Box 73, El Tarf 36000, Algeria · Mathematical Modeling and Numerical Simulation Laboratory (LAM2SIN), Faculty of Technology, Badji Mokhtar University, P.O. Box 12, Annaba 23000, Algeria
GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization
May 12, 2026Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha +1LLM Inference Efficiency Constrained Optimization
Manipal University Jaipur · TCS Research

Feb 25, 2026J/K move · Enter open · S save

Sustainable LLM Inference using Context-Aware Model Switching

Yuvarani, Akashdeep Singh, Zahra Fathanah, Salsabila Harlen+2

Faculty of Computer Science and Information Technology, Universiti Malaya, 50603 Kuala Lumpur, Malaysia

LLM Inference Efficiency LLM Inference Energy-Efficient ML LLM Routing

Open paper AskPDF arXiv

Large language models have become central to many AI applications, but their growing energy consumption raises serious sustainability concerns. A key limitation in current AI deployments is the reliance on a one-size-fits-all inference strategy where most systems route every request to the same large model, regardless of task complexity, leading to substantial and unnecessary energy waste. To address this issue, we propose a context-aware model switching approach that dynamically selects an appropriate language model based on query complexity. The proposed system uses a Context-Aware Model Switching for Energy-Efficient LLM Inference that combines caching for repeated queries, rulebased complexity scoring for fast and explainable decisions, machine learning classification to capture semantic intent, and a user-adaptive component that learns from interaction patterns over time. The proposed architecture was evaluated using real conversation workloads and three open-source language models (Gemma3 1B, Gemma3 4B and Qwen3 4B) with different computational costs, measuring energy consumption (via NVML GPU power telemetry), response latency, routing accuracy, and output quality (BERTScore F1) to reflect real-world usage conditions. Experimental results show that the model switching approach can reduce energy consumption by up to 67.5% compared to always using the largest model while maintaining a response quality of 93.6%. In addition, the response time for simple queries also improved significantly by approximately 68%. These results show that model switching inference offers a practical and scalable path toward more energy-efficient and sustainable AI systems, demonstrating that significant efficiency gains can be achieved without major sacrifices in response quality.

Abstract

LLM Inference Efficiency

Explore similar work

Sustainable LLM Inference using Context-Aware Model Switching

Environmental Impact of Generative and Agentic AI: An in-Depth Analysis and Green Solutions

GAR: Carbon-Aware Routing for LLM Inference via Constrained Optimization