SkillEarth: Teaching General-Purpose Agents to Act on Earth via Skill Evolution
Authors: Haojun Tang, Wenda Zhao, Xiang Zhou, Qiwei Lin, Lida Huang, Yan Wang
Organizations: Institute for AI Industry Research (AIR), Tsinghua University · Dalian University of Technology · Beijing Institute of Radio Measurement
Recently developed remote sensing (RS) agents execute real-world tasks with complex workflows through multi-step tool invocation. These agents, however, typically rely on predefined expertise or parameter updates. They thus require updating expert knowledge or retraining models whenever new tools, tasks, or execution patterns emerge, limiting their continual adaptability in dynamic Earth environments. In this work, we seek to equip general-purpose agents with RS tool-call knowledge that can be automatically extracted from demonstrated trajectories without expert intervention or parameter updates. Specifically, we propose SkillEarth, a self-evolving skill extraction method that externalizes RS tool-call knowledge as reusable, retrievable, and evolvable skill files and refines them through observation-grounded evolution. Unlike existing automatic skill extraction methods that rely on outcome-based reflection to refine skills holistically, SkillEarth grounds skill evolution in real tool-chain execution and combines dual-judge feedback with a grounded observation routing graph for precise component-level refinement. The resulting skills are generalizable, reliable, and transferable: they improve overall RS tool-call performance across three frontier LLMs and four open-source LLMs, are refined through real tool execution feedback, and support cross-model reuse without retraining. Experiments on OpenEarth-Bench and Earth-Bench demonstrate that SkillEarth achieves better tool selection, argument correctness, execution-order consistency, and task completion than existing RS agents and skill extraction methods.
Figures & tables
Figure 2: Comparison of (a) existing skill extraction with outcome-based holistic refinement and (b) SkillEarth with component-level refinement via observation-grounded evolution.
Figure 3: Overview of SkillEarth. (a) The Extractor constructs component-wise skills by overwriting templates with LLM-inferred components. (b) The Evolver validates and refines skills using dual-judge feedback and GOR graphs. (c) The Decider merges, discards, or adds skills to the skill bank. (d) During application, the Executor uses retrieved skills to generate final answers.
Figure 4: Qualitative analysis of a skill before and after evolution. (a) The initial extracted skill contains over-specified task planning and ambiguous observation routing. (b) The evolved skill adopts generalizable task planning and precise output-to-argument routing.
Figure 5: Qualitative comparison of SkillEarth with state-of-the-art RS agents. We visualize their tool-calling trajectories, corresponding tool outputs, and final images showing the requested detection of the smallest airplane. OpenEarthAgent, RSMeM, and RS-Agent produce incorrect results, while Earth-Agent obtains the correct result but follows an incorrect tool-calling trajectory. In contrast, SkillEarth correctly identifies the smallest airplane by computing the areas of all six candidate boxes through six Calculator calls.
Method
EP
FT
Step-by-Step
End-to-End
Ins. ↑
Tool. ↑
ArgN. ↑
ArgV. ↑
Summ. ↑
Per. ↑
Op. ↑
Logic. ↑
GIS. ↑
AnyOr. ↑
SameO. ↑
Uni. ↑
Ans. ↑
Gen. ↑
OpenEarthAgent (ECCV’26) [ 26 ]
✗
✗
97.34
86.94
84.12
33.55
83.28
13.86
20.95
18.35
71.82
16.00
14.71
21.47
13.72
15.86
Earth-Agent (ICLR’26) [ 8 ]
✗
✗
99.98
90.84
87.31
33.37
88.70
31.26
14.67
11.94
51.58
2.48
2.31
4.19
12.82
19.31
RSMeM (ACL’26) [ 33 ]
✓
✗
99.20
87.97
85.06
33.05
84.92
31.20
7.78
15.08
53.56
9.41
8.98
11.46
18.19
33.10
RS-Agent (Sci. China Inf. Sci.’26) [ 35 ]
✓
✗
100.00
43.66
40.17
11.04
88.31
28.52
19.67
18.72
12.67
3.93
3.93
5.82
9.46
37.93
SkillEarth
✗
✗
99.07
91.04
86.48
36.17
92.67
35.00
33.06
28.73
81.85
25.32
25.15
27.89
19.59
24.14
Table 1: Comparison with state-of-the-art RS agents using the Qwen3-4B-Instruct on OpenEarth-Bench. EP and FT denote expert priors and fine-tuning, respectively. End-to-End metrics include F 1 scores (Per., Op., Logic., GIS), tool order (AnyOr., SameO., Uni.), and accuracy (Ans., Gen.). The top three results are highlighted in red , blue , and orange , respectively.
Method
EP
FT
Automatic Planning (AP)
Instruction Following (IF)
Step-by-Step
End-to-End
Step-by-Step
End-to-End
AnyOr. ↑
InOr. ↑
Exact. ↑
Param. ↑
Eff. ↓
Acc. ↑
AnyOr. ↑
InOr. ↑
Exact. ↑
Param. ↑
Eff. ↓
Acc. ↑
OpenEarthAgent (ECCV’26) [ 26 ]
✗
✗
46.55
39.01
28.59
18.98
2.7099
18.15
50.21
41.68
30.75
20.75
2.7778
19.76
Earth-Agent (ICLR’26) [ 8 ]
✗
✗
20.47
1.32
0.00
0.00
3.9747
15.73
29.69
10.89
8.63
4.21
5.4554
10.08
RSMeM (ACL’26) [ 33 ]
✓
✗
50.43
33.25
7.68
7.51
1.1703
21.46
40.72
23.27
13.93
1.16
0.4839
21.46
HiRS-Agent (ACM MM’26) [ 17 ]
✓
✓
50.95
41.10
31.67
19.54
2.5059
43.95
53.79
46.34
34.64
20.81
3.3013
45.56
Table 2: Comparison with state-of-the-art RS agents with the Qwen3-4B-Instruct on Earth-Bench under AP and IF settings. The top three results are highlighted in red , blue , and orange , respectively.
Method
Step-by-Step
End-to-End
Ins. ↑
Tool. ↑
ArgN. ↑
ArgV. ↑
Summ. ↑
Per. ↑
Op. ↑
Logic. ↑
GIS. ↑
AnyOr. ↑
SameO. ↑
Uni. ↑
Ans. ↑
Gen. ↑
Trace2Skill (arXiv’26) [ 19 ]
98.88
89.89
85.61
36.35
84.43
13.21
30.56
22.97
40.33
2.40
2.40
3.51
9.67
14.48
AutoSkill (arXiv’26) [ 36 ]
98.78
83.60
79.61
38.35
87.31
17.97
24.88
20.02
35.96
3.85
3.76
4.96
8.31
9.66
SkillX (arXiv’26) [ 31 ]
99.03
90.37
85.62
34.19
84.68
15.80
29.72
20.12
72.62
17.11
16.77
20.87
20.12
17.24
SkillRevise (EMNLP’26) [ 15 ]
98.51
88.30
84.07
33.60
83.83
13.50
25.26
21.31
35.50
2.40
2.22
3.25
6.63
14.48
SkillEarth
99.07
91.04
86.48
36.17
92.67
35.00
33.06
28.73
81.85
25.32
25.15
27.89
19.59
24.14
Table 3: Comparison with skill extraction methods using the Qwen3-4B-Instruct on OpenEarth-Bench. End-to-End metrics include F 1 scores (Per., Op., Logic., GIS), tool order (AnyOr., SameO., Uni.), and accuracy (Ans., Gen.). The top three results are highlighted in red , blue , and orange , respectively.
Method
Automatic Planning (AP)
Instruction Following (IF)
Step-by-Step
End-to-End
Step-by-Step
End-to-End
AnyOr. ↑
InOr. ↑
Exact. ↑
Param. ↑
Eff. ↓
Acc. ↑
AnyOr. ↑
InOr. ↑
Exact. ↑
Param. ↑
Eff. ↓
Acc. ↑
Trace2Skill (arXiv’26) [ 19 ]
46.14
33.03
26.34
20.48
1.8854
20.81
49.55
39.39
32.53
24.29
2.0532
24.60
AutoSkill (arXiv’26) [ 36 ]
43.56
28.63
17.21
13.12
1.6711
18.55
48.34
35.44
25.52
18.01
1.8344
22.18
SkillX (arXiv’26) [ 31 ]
51.54
42.95
33.43
20.66
1.9922
20.19
58.00
48.87
44.17
27.27
1.9772
18.15
SkillRevise (EMNLP’26) [ 15 ]
46.42
34.68
24.93
19.22
1.7609
22.18
47.65
35.79
30.03
21.86
1.8034
22.58
Table 4: Comparison with skill extraction methods using the Qwen3-4B-Instruct on Earth-Bench under AP and IF settings. The top three results are highlighted in red , blue , and orange , respectively.
Model
Setting
Param.
Step-by-Step
End-to-End
Inst. ↑
Tool. ↑
ArgN. ↑
ArgV. ↑
Summ. ↑
Per. ↑
Op. ↑
Logic. ↑
GIS. ↑
AnyOr. ↑
SameO. ↑
Uni. ↑
Ans. ↑
Gen. ↑
Frontier LLMs
GPT-5.6-Terra [ 20 ]
w/o skill
–
96.35
81.88
79.34
37.95
96.88
30.99
46.40
6.76
92.63
45.17
45.00
47.13
50.82
35.86
w/ skill
–
98.46
88.35
84.99
43.07
97.30
44.37
65.59
20.50
95.59
51.84
51.33
55.18
53.91
51.72
Doubao-Seed-2.0-Mini [ 24 ]
w/o skill
–
99.83
88.72
82.46
36.12
94.02
34.74
28.19
19.49
87.18
33.96
33.62
38.84
36.63
56.55
w/ skill
–
99.86
89.75
86.50
39.46
96.13
43.58
20.69
27.23
89.09
45.51
44.82
49.70
43.78
51.03
Table 5: Skill generalizability across different LLMs on OpenEarth-Bench. All models use a maximum output token budget of 1024. End-to-End metrics include F 1 scores (Per., Op., Logic., GIS), tool order (AnyOr., SameO., Uni.), and accuracy (Ans., Gen.). bold indicates the best result for each model. For each metric, the top three models are highlighted with red , cyan , and green backgrounds, respectively.
Setting
Step-by-Step
End-to-End
Tool.
ArgN.
Summ.
Ans.
Gen.
Baseline
90.33
85.36
92.01
18.35
18.62
+ Tool Judge
90.92
85.70
92.51
18.42
20.09
+ Semantic Judge
90.81
85.99
92.66
18.49
23.45
+ GOR Graph ( Ours )
91.04
86.48
92.67
19.59
24.14
Table 6: Ablation study of SkillEarth. The top three results are highlighted in red , blue , and orange , respectively. Complete results are provided in Tabs. 14 and 13 of the Supp. 12 .
Task types
Task types (continued)
Task types (continued)
Task type
Traj.
Original
Selected
Task type
Traj.
Original
Selected
Task type
Traj.
Original
Selected
⟨missing⟩
5
13
13
complex_type3
1
21
10
type14_SAR
1
87
10
b1_type1
1
65
10
complex_type4
1
46
10
type15
1
99
10
b1_type2
1
63
10
complex_type5
1
9
9
type19
3
395
30
b1_type3
1
63
10
complex_type6
1
9
9
type1_SAR
3
110
30
b1_type4
1
58
10
complex_type7
1
1
1
type25
4
739
40
Table 7: Training-set composition after trajectory-stratified sampling. “Traj.” denotes the number of distinct tool-call trajectories within a task type. For each type–trajectory combination, up to 10 samples were retained. Original denotes the number of available training samples in each task type, while Selected denotes the number of samples used for SkillBank construction.
Task types
Number
Proportion (%)
Temperature Monitoring
44
17.74
Weather Forecasting
11
4.44
Climate Analysis
20
8.06
Water Management
22
8.87
Pollution Regulation
14
5.65
Vegetation Monitoring
28
11.29
Table 8: Task types and their proportions in Earth-Bench.
Category
Backbone
Usage in SkillEarth
Temp.
Max Tokens
Experimental LLMs
GPT-5.6-Terra [ 20 ]
Extractor , Executor , Refiner , Decider , Merger
0.0
1024 / 16384 †
Doubao-Seed-2.0-Mini [ 24 ]
DeepSeek-V4-Flash [ 7 ]
Mistral-v0.3-Instruct [ 10 ]
DeepSeek-R1 [ 6 ]
Qwen2.5-7B-Instruct [ 22 ]
Table 9: LLM configurations used in SkillEarth experiments. Temp. denotes the temperature parameter of LLMs.
Stage
Parameter
Setting
SkillBank Construction
Max. tool rounds ( nt )
5
Max. semantic rounds ( ns )
2
Acceptance rule
Text: s>0.6
Image: exist=1
Skill Retrieval
Encoder
Hashing
Dimension
2048
Table 10: Hyperparameter settings for SkillBank construction and retrieval. Here, exist=1 indicates that the required image output is successfully generated.
Evaluation Model
Setting
Skill Source
Inst. ↑
Tool. ↑
ArgN. ↑
ArgV. ↑
Summ. ↑
Qwen3-4B-Instruct
w/o skill
–
98.47
87.70
84.33
33.67
92.55
w/ skill
Doubao-Seed-2.0-Mini
98.66
91.31
86.97
37.12
94.46
w/ skill
DeepSeek-V4-Flash
98.47
90.26
86.94
37.07
93.22
w/ skill
GPT-5.6-Terra
98.80
90.53
86.82
36.20
93.04
DeepSeek-V4-Flash
w/o skill
–
99.78
84.92
78.90
40.32
96.64
w/ skill
Qwen3-4B-Instruct
99.92
89.08
81.59
43.41
96.70
Table 11: Step-by-Step cross-model skill transfer on OpenEarth-Bench. “Skill Source” denotes the model used to extract the transferred skills. The best, second-best, and third-best results within each evaluation model are highlighted in red , blue , and orange , respectively.
Evaluation Model
Setting
Skill Source
F 1 Scores
Tool Order
Accuracy
Per. ↑
Op. ↑
Logic. ↑
GIS. ↑
AnyOr. ↑
SameO. ↑
Uni. ↑
Ans. ↑
Gen. ↑
Qwen3-4B-Instruct
w/o skill
–
13.31
12.54
11.80
76.42
15.23
14.63
18.48
16.94
7.59
w/ skill
Doubao-Seed-2.0-Mini
33.14
33.58
23.82
80.23
23.95
23.78
27.20
21.26
23.45
w/ skill
DeepSeek-V4-Flash
23.98
25.58
20.09
77.53
22.07
21.73
24.12
18.92
18.62
w/ skill
GPT-5.6-Terra
29.26
31.69
18.48
76.21
20.62
20.36
22.93
17.32
22.76
DeepSeek-V4-Flash
w/o skill
–
26.05
38.51
13.11
82.27
50.90
50.21
56.54
53.41
55.86
Table 12: End-to-End cross-model skill transfer on OpenEarth-Bench. “Skill Source” denotes the model used to extract the transferred skills. The best, second-best, and third-best results within each evaluation model are highlighted in red , blue , and orange , respectively.
Setting
Ins. ↑
Tool. ↑
ArgN. ↑
ArgV. ↑
Summ. ↑
Baseline
98.91
90.33
85.36
35.66
92.01
+ Tool Judge
98.85
90.92
85.70
35.29
92.51
+ Semantic Judge
98.76
90.81
85.99
35.95
92.66
+ GOR Graph ( Ours )
99.07
91.04
86.48
36.17
92.67
Table 13: Ablation study of SkillEarth on OpenEarth-Bench under the Step-by-Step setting using Qwen3-4B-Instruct. The top three results are highlighted in red , blue , and orange , respectively.
Setting
F 1 scores
Tool Order
Accuracy
Per. ↑
Op. ↑
Logic. ↑
GIS. ↑
AnyOr. ↑
SameO. ↑
Uni. ↑
Ans. ↑
Gen. ↑
Baseline
31.71
27.45
24.38
80.04
23.61
23.44
26.60
18.35
18.62
+ Tool Judge
32.43
36.44
25.78
81.02
24.12
23.78
26.60
18.42
20.09
+ Semantic Judge
33.52
31.58
28.56
81.24
24.24
24.15
26.80
18.49
23.45
+ GOR Graph ( Ours )
35.00
33.06
28.73
81.85
25.32
25.15
27.89
19.59
24.14
Table 14: Ablation study of SkillEarth on OpenEarth-Bench under the End-to-End setting using Qwen3-4B-Instruct. The top three results are highlighted in red , blue , and orange , respectively.
Figure 6: Qualitative comparison of SkillEarth with existing skill extraction methods. We visualize the tool-calling trajectories and outputs of Qwen3-4B-Instruct on an NBR-based burn severity analysis task. Existing methods fail due to missing raster layers, while SkillEarth successfully completes the task by following the correct tool execution sequence.
Figure 7: End-to-End generalization of SkillEarth across seven LLMs on OpenEarth-Bench. We compare performance with and without skill guidance across nine evaluation metrics. The values above the bars indicate performance changes in percentage points.
Agent skills, which consist of reusable strategies that guide agent reasoning and action, have shown strong potential for improving model capability at inference time. However, current skill construction methods treat the problem as one-shot extraction, overlooking a fundamental tension: a skill tailored to the specific task fails to transfer, while the abstracted skill often provides insufficient guidance. We attribute this fragility to the absence of explicit mechanisms for skill specification and generalization. To address this gap, we introduce SkillComposer, a framework that decomposes skill construction into three learnable operations: create, improve, and merge. Trained via systematic rejection sampling recipe, SkillComposer enables language models to self-evolve skills at inference time and supports three deployment modes: offline for building generalized libraries, online for task-specific refinement, and hybrid for combining both. Comprehensive experiments on τ2-Bench, LiveCodeBench v6, and AppWorld show that SkillComposer consistently outperforms baselines. Our SkillComposer-4B improves a 27B executor by up to +4.5 on agent tasks and +3.4 on code tasks, while generalizing across domains and task types unseen during training. Analysis reveals that merge and improve address orthogonal quality dimensions and that skill composition is a transferable meta-ability, providing a practical recipe for skill-augmented inference.
Qi Zhang, Zhaopeng Feng, Xiaonan Shi +8
1Zhejiang University · 2Tongyi Lab · 3National University of Singapore
Reusable skills, which encapsulate the procedural knowledge required to solve real-world professional tasks, offer LLM-based agents a path toward self-evolution in expert domains. Existing self-evolving skill methods construct skills internally from the model's parametric knowledge or trajectories, and are therefore bounded by what the model already knows. However, the domain conventions and standard procedures underlying professional skills often lie beyond this boundary and are hard to elicit from the agent alone. To address this issue, we therefore propose a novel framework, Search2Skill, that automatically identifies the agent's capability gaps, searches external sources to address them, and distills the retrieved evidence into structured, reusable skills. Specifically, Search2Skill is optimized by a rubric-based reinforcement learning scheme that jointly improves when to search, how to search, and how to generate skills. Experiments on eight expert-level domains from three benchmarks show that Search2Skill consistently outperforms both search-augmented and trajectory-based skill-learning baselines under both streaming and held-out evaluation protocols. Further analyses show that the gains arise from skill abstraction rather than raw retrieved evidence, and that the acquired skills transfer across model scales.
Muyang Ye, Tian Lan, Feihu Jiang +10
Zhejiang University · Alibaba Group · Institute of Artificial Intelligence, Xiamen University +1
A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.
Yaorui Shi, Yuxin Chen, Zhengxi Lu +6
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2