RS-Claw-Evolution: Environment-Feedback-Driven Evolution for Lightweight Remote Sensing Agents in Long-Horizon Tasks

Kai Ouyang; Dongyang Hou; Liangtian Liu; Zeyuan Wang; Ziyu Li; Chengfu Liu; Zichao Tang; Xuezhi Cui

cs.LGSep 6, 2026

RS-Claw-Evolution: Environment-Feedback-Driven Evolution for Lightweight Remote Sensing Agents in Long-Horizon Tasks

Authors: Kai Ouyang, Dongyang Hou, Liangtian Liu, Zeyuan Wang, Ziyu Li, Chengfu Liu, Zichao Tang, Xuezhi Cui,

Organizations: School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · School of Earth Sciences and Spatial Information Engineering, Hunan University of Science and Technology, Xiangtan 411201, China · Sanya Institute, Hunan University of Science and Technology, Sanya 572024, China · School of Resources and Environment, University of Electronic Science and Technology of China, Chengdu 611700, China

Abstract

Large language model-driven remote sensing (RS) agents offer a promising approach to automating geospatial analysis. However, lightweight RS agents based on compact language models struggle with multi-step interactive tasks due to loss of long-horizon states, inefficient environmental feedback utilization, and sparse optimization signals. We propose RS-Claw-Evolution, an environment-feedback-driven framework that progressively improves lightweight agents through three stages. Interaction evolution uses executable code to control observations, maintain intermediate states, and reduce context redundancy. Experience evolution combines failure-aware trajectory generation with error-turn masking to learn from informative failure-recovery experiences without imitating faulty actions. Decision evolution uses reinforcement learning with multi-dimensional environment rewards and turn-level advantage protection to optimize tool-use behaviors and improve credit assignment in long sequences. On Earth-Bench, the optimized Qwen3-4B-based agent achieves 65.9% accuracy in Autonomous Planning mode, outperforming the untrained Qwen3-32B baseline (43.8%) and DeepSeek-V3.1 (60.8%), while approaching GPT-5 (71.6%). These results demonstrate that learning from environmental feedback can improve lightweight agents and narrow their performance gap with larger models in long-horizon RS tasks.

Figures & tables

Fig. 1: Comparison between the original RS-Claw paradigm and the proposed RS-Claw-Evolution framework. The original RS-Claw paradigm treats environmental feedback as execution information during inference, while RS-Claw-Evolution further exploits environmental feedback as evolution signals to progressively improve lightweight agents through three stages: Interaction Evolution via CodeAct-based interaction, Experience Evolution via failure-aware experience learning, and Decision Evolution via feedback-driven policy optimization.

Fig. 2: Overview of the Experience Evolution stage in RS-Claw-Evolution: feedback-driven SFT for lightweight RS agents. (Top) Hint-guided Failure-aware Trajectory Generation extracts task-level hints through hindsight analysis of failed trajectories with reference to successful code trajectories. These hints guide teacher-agent re-exploration, followed by rejection sampling to retain successful trajectories containing intermediate error feedback and recovery behaviors. (Bottom) Error-turn Masking Training removes explicit hints from student training inputs and masks the loss on erroneous action turns while retaining error feedback and subsequent recovery behaviors, enabling lightweight agents to learn error diagnosis and recovery without imitating faulty actions.

Fig. 3: Overview of the Decision Evolution stage in RS-Claw-Evolution: feedback-driven GRPO optimization for long-horizon RS agents. Compared with GRPO using sparse trajectory-level rewards, the proposed approach incorporates multi-dimensional process rewards derived from environmental feedback for finer-grained policy optimization. Furthermore, TAP selectively masks misleading turn-level advantages in successful trajectories containing erroneous exploration turns, reducing erroneous positive reinforcement while protecting recovery behaviors from inappropriate negative penalties. Together, these mechanisms support more reliable and efficient decision-making in long-horizon tool-use tasks.

Model	Paradigm	Accuracy		Tool-Any-Order		Tool-In-Order
Model	Paradigm	AP	IF	AP	IF	AP	IF
Qwen3-4B-Instruct	Code	31.8	35.8	47.6	52.8	40.1	45.8
Qwen3-4B SFT	Code	41.5	44.3	58.6	60.9	49.4	51.8
Qwen3-4B RL (Ours)	Code	65.9	65.9	69.1	70.1	58.9	58.4
Qwen3-8B	Code	19.3	22.7	49.6	50.9	41.1	44.2
Qwen3-8B SFT	Code	46.6	44.3	61.4	63.6	52.2	53.2

TABLE I: Overall Performance Comparison of Language Models with Varying Scales on Earth-Bench. "Code" indicates the programmatic interaction paradigm utilized in our method, while "JSON" represents the traditional structured tool-calling paradigm used in the original RS-Claw framework.

Fig. 4: Comparison of context efficiency between Code and JSON interaction paradigms. The left and right panels respectively show the average Action Tokens and Observation Tokens consumed in a single trajectory by different language models (DeepSeek-V3.1, GPT-5, and Qwen3-32B). The results indicate that compared to the traditional structured JSON tool invocation paradigm, the programmatic interaction paradigm (Code) adopted in this paper effectively reduces context overhead in both AP and IF modes.

Fig. 5: Impact of the Hindsight Hint guidance mechanism on the trajectory generation success rate of 12 training tasks. The solid blue line and dashed red line represent the exploration success rates of the teacher agent with and without the Hindsight Hint guidance, respectively.

Scale	SFT Strategy	Accuracy	Tool-Any-Order	Tool-In-Order	EIR
4B	Before Training	31.8	47.6	40.1	45.8
	Oracle	33.5	54.2	44.8	57.4
	Unmasked	37.5	58.9	48.6	36.9
	Masked (Ours)	41.5	58.6	49.4	31.6
8B	Before Training	19.3	49.6	41.1	53.1
	Oracle	35.2	56.7	47.5	57.8

TABLE II: Ablation Study of SFT Strategies for Agent Error Recovery Across Model Scales. Oracle, Unmasked, and Masked denote error-free expert trajectory SFT, full-trajectory SFT, and the proposed error-turn masking strategy, respectively.

Variant	Accuracy	Tool-Any-Order	Tool-In-Order	Avg Turns	EIR
SFT Only	41.5	58.6	49.4	17.426	31.6
Vanilla GRPO	58.5	64.9	53.7	11.688	10.3
w/o MEFR	61.9	68.1	58.1	12.920	8.0
w/o TAP	63.1	68.6	58.6	13.068	7.7
Ours	65.9	69.1	58.9	12.761	6.1

TABLE III: Ablation Study of RL Components Based on the Qwen3-4B-Instruct Model After the Proposed SFT Phase.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Task ID	Application Domain	Workflow Type
3	Vegetation and Water Stress	Composite Analysis
43	Vegetation and Water Stress	Threshold Analysis
14	Land Surface Thermal	Temporal Trend
39	Land Surface Thermal	Threshold Analysis
77	Land Surface Thermal	Dual-object Difference
72	Land Surface Thermal	Composite Analysis

Appendix

TABLE S1: Selected Training Tasks.

Hyperparameter	Value
Training epochs	2
Global batch size	4
Max tokens per GPU	20,480
Max sample tokens	20,480
Learning rate	$5\times 10^{-6}$
Minimum learning rate	$5\times 10^{-7}$

Appendix

TABLE S2: Hyperparameters for Supervised Fine-Tuning (SFT).

Hyperparameter	Value
Training epochs	5
Learning rate	$1\times 10^{-6}$
LR schedule	Constant
Optimizer	Adam ( $\beta_{1}=0.9,\beta_{2}=0.98$ )
Weight decay	0.1
KL loss coefficient	0.001

Appendix

TABLE S3: Hyperparameters for Reinforcement Learning (GRPO).

Metric	Unmasked SFT	Masked SFT (Ours)
Turn-level Statistics
Total Agent Turns	17,816	17,816
Trainable Turns	17,816	14,929
Masked Turns	0	2,887
Turn Retention Ratio	100%	83.80%
Turn Masking Ratio	0%	16.20%

Appendix

TABLE S4: Turn-Level and Token-Level Masking Statistics.

Model	Paradigm	Accuracy		Efficiency		Tool-Any-Order		Tool-In-Order		Tool-Exact-Match		Parameters
Model	Paradigm	AP	IF	AP	IF	AP	IF	AP	IF	AP	IF	AP	IF
Qwen3-4B-Instruct	Code	31.8	35.8	3.030	3.904	47.6	52.8	40.1	45.8	35.4	37.8	23.8	24.2
Qwen3-4B SFT	Code	41.5	44.3	4.650	4.356	58.6	60.9	49.4	51.8	42.9	42.6	23.9	24.8
Qwen3-4B RL (Ours)	Code	65.9	65.9	4.917	6.446	69.1	70.1	58.9	58.4	48.6	48.2	24.1	24.0
Qwen3-8B	Code	19.3	22.7	3.957	3.907	49.6	50.9	41.1	44.2	36.9	37.8	23.8	24.8
Qwen3-8B SFT	Code	46.6	44.3	4.242	5.158	61.4	63.6	52.2	53.2	44.3	42.7	25.3	25.1

Appendix

TABLE S5: Comprehensive Performance Evaluation Results. All metrics except Efficiency are reported as percentages.

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 711 papers

RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents
May 13, 2026Liangtian Liu, Zeyuan Wang, Ziyu Li +8Remote Sensing Image Understanding Tool-Using Agents
School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · School of Resources and Environment, University of Electronic Science and Technology of China, Xian 710071, China · School of Earth Sciences and Spatial Information Engineering, Hunan University of Science and Technology, Xiangtan 411201, China +1
RingMoClaw: An Experience-Inspired Multi-Agent Framework for Self-Evolving Research in Remote Sensing
Sep 1, 2026Kaiyue Kang, Qixuan He, Peijin Wang +9Remote Sensing Image Understanding Memory-Augmented Agents
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China · University of Chinese Academy of Sciences, Beijing 100190, China +1
HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving
Aug 31, 2026Boyang Mu, Zhiwei Wei, Mugen Peng +1Multi-Agent Collaboration Remote Sensing
Beijing University of Posts and Telecommunications State Key Laboratory of Networking and Switching Technology Beijing, China · Hunan Normal University School of Geographic Sciences Changsha, Hunan, China

May 13, 2026J/K move · Enter open · S save

RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents

Liangtian Liu, Zeyuan Wang, Ziyu Li, Kai Ouyang+7

School of Geosciences and Info-Physics, Central South University, Changsha 410083, China · School of Resources and Environment, University of Electronic Science and Technology of China, Xian 710071, China · School of Earth Sciences and Spatial Information Engineering, Hunan University of Science and Technology, Xiangtan 411201, China+1

Remote Sensing Image Understanding Tool-Using Agents Long-Horizon LLM Agents

Open paper AskPDF arXiv

The rise of multi-modal large language models (MLLMs) is shifting remote sensing (RS) intelligence from "see" to "action", as OpenClaw-style frameworks enable agents to autonomously operate massive RS image-processing tools for complex tasks. Existing RS agents adopt a passive selection paradigm for tool invocation, relying on either full tool registration (Flat) or retrieval-augmented generation (RAG). However, in the massive and multi-source heterogeneous RS tool ecosystem, such passive mechanisms struggle to dynamically balance "context load" and "toolset completeness" throughout task reasoning, thus exhibiting inherent limitations: full tool registration triggers context space deficits during long-horizon tasks, whereas RAG retrieval may omit critical tools in essential steps. To overcome these bottlenecks, this paper redefines tool selection by arguing that the agent should act as an active explorer within the tool space. Based on this perspective, we propose RS-Claw, a novel RS agent architecture. By leveraging Skill encapsulation technology at the tool end, this architecture hierarchically structures tool descriptions, enabling the agent to execute on-demand sequential decision-making: initially selecting relevant skill branches by reading only tool summaries, then dynamically loading detailed descriptions, and ultimately achieving precise invocation. This active paradigm not only significantly liberates the agent's context space but also effectively ensures the accurate hit rate of critical tools during long-horizon reasoning. Systematic experiments on the Earth-Bench benchmark demonstrate that RS-Claw's active exploration mechanism effectively filters semantic noise and substantially frees up reasoning space, achieving an input token compression ratio of up to 86%, and comprehensively outperforming existing Flat and RAG baselines across complex reasoning evaluations.

Abstract

LLM Agents

Explore similar work

RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents

RingMoClaw: An Experience-Inspired Multi-Agent Framework for Self-Evolving Research in Remote Sensing

HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving