cs.LGApr 14, 2026
SaveFrom Imitation to Discrimination: Progressive Curriculum Learning for Robust Web Navigation
Organizations: Beijing Jiaotong University · Beihang University
Abstract
Text-based web agents offer computational efficiency for autonomous web navigation, yet developing robust agents remains challenging due to the noisy and heterogeneous nature of real-world HTML. Standard Supervised Fine-Tuning (SFT) approaches fail in two critical dimensions: they lack discrimination capabilities to reject plausible but incorrect elements in densely populated pages, and exhibit limited generalization to unseen website layouts. To address these challenges, we introduce the Triton dataset (590k instances) and a progressive training curriculum. Triton is constructed via Structural-Semantic Hard Negative Mining, which explicitly mines topologically similar distractors, and a Dual-Agent Consensus pipeline that synthesizes diverse cross-domain tasks with strict verification. Building upon this foundation, our progressive curriculum produces three models: Triton-SFT-32B for basic imitation, Triton-ORPO-32B for robust discrimination via Odds Ratio Preference Optimization, and Triton-GRPO-32B for long-horizon consistency through Group Relative Policy Optimization. Empirical evaluation on Mind2Web demonstrates that Triton-GRPO-32B achieves state-of-the-art performance among open-source models with 58.7% Step Success Rate, surpassing GPT-4.5 (42.4%) and Claude-4.5 (41.4%) by over 16%, validating that specialized data curriculum outweighs raw parameter scale for web navigation.
Figures & tables
Figure 1: Standard imitation learning hallucinates on structurally similar elements (incorrectly selecting iPhone 17 Pro when instructed to buy iPhone 16 Pro), while Triton trained with progressive curriculum (ORPO for discrimination + GRPO for consistency) achieves target identification and multi-step task completion.
Figure 2: Overview of our proposed framework. (Left) We construct the Triton dataset by mining topological hard negatives and synthesizing cross-domain tasks via Dual-Agent Consensus. (Middle) This data fuels a progressive curriculum that evolves the model from imitation (SFT) to discrimination (ORPO) and consistency (GRPO), (Right) enabling robust execution in multi-step environments.
| Component | Method | # Samples |
| Mind2Web (Base) | - | 7k |
| + Hard Neg. | DOM-Tree Mining | 140k |
| + Rejection | Perturbation | 153k |
| In-Domain Total | 300k | |
| WebSight (Synth.) | Dual-Agent | 290k |
| Triton Total | 590k |
Table 1: Composition of the Triton Dataset
Figure 3: Visualization of semantic diversity via t-SNE. While the original Mind2Web data (Blue) forms dense, domain-specific clusters, our synthetic WebSight data (Green) spans a broader semantic space. This complementary distribution demonstrates that Triton effectively fills the "generalization gaps" left by standard behavioral cloning datasets.
| Split | # Tasks | # Domains | # Websites | Avg # Actions |
| Train | 1,009 | 17 | 73 | 7.7 |
| Cross-Task | 177 | 17 | 64 | 7.6 |
| Cross-Website | 142 | 9 | 10 | 7.2 |
| Cross-Domain | 694 | 13 | 53 | 5.9 |
Table 2: Statistics of the Mind2Web dataset. The benchmark is divided into three test splits to evaluate generalization difficulty. Note that the Cross-Domain split contains the largest number of tasks but the shortest average trajectory length.
| Model | Size | All | Cross-Domain | Cross-Task | Cross-Website | ||||||||||||||||
| Score | SR | Score | SR | Ele | Op | M.Ele | M.Op | Score | SR | Ele | Op | M.Ele | M.Op | Score | SR | Ele | Op | M.Ele | M.Op | ||
| Closed-Source LLMs | |||||||||||||||||||||
| Claude-4-5-noThink | 58.8 | 41.4 | 58.1 | 41.8 | 48.3 | 67.0 | 49.9 | 67.2 | 61.5 | 43.7 | 52.0 | 68.3 | 55.2 | 70.4 | 56.9 | 38.7 | 46.8 | 65.1 | 48.7 | 67.0 | |
| Gemini-2.0-Flash | 53.1 | 35.0 | 52.6 | 35.2 | 43.1 | 61.2 | 44.5 | 61.5 | 55.7 | 37.1 | 46.5 | 63.4 | 48.8 | 64.2 | 51.0 | 32.8 | 41.2 | 59.5 | 43.1 | 60.2 | |
| Gemini-2.5-Flash-preview-04-17 | 55.3 | 38.0 | 54.9 | 38.4 | 45.4 | 63.5 | 46.8 | 63.9 | 57.7 | 40.1 | 48.2 | 65.1 | 50.5 | 66.8 | 53.3 | 35.6 | 43.5 | 61.8 | 45.2 | 62.5 | |
| Gemini-2.5-pro-preview-05-06 | 59.1 | 41.5 | 58.3 | 41.6 | 48.5 | 67.2 | 50.1 | 67.5 | 61.7 | 43.9 | 52.2 | 68.5 | 55.4 | 70.6 | 57.2 | 39.1 | 47.1 | 65.4 | 49.0 | 67.2 | |
Table 3: Main results on the Mind2Web benchmark. We report the Composite Score and Step Success Rate (SR) across three generalization splits. Triton-GRPO-32B significantly outperforms all open-source baselines (including the 480B model) and surpasses top-tier proprietary models like GPT-4.5 and Claude-4.5 in Step SR, demonstrating robust generalization despite its smaller parameter size.
Figure 4: Training dynamics analysis. Performance trajectories of the Full Curriculum versus variants with insufficient SFT foundations. Shaded areas denote alignment stages. The persistent gap confirms that ORPO and GRPO act as behavioral refiners rather than knowledge injectors, and thus cannot recover from a weak SFT baseline.
| Model Variant | SFT Triton Data | RL Setting | Overall | Cross-Domain | Cross-Task | Cross-Website | |||||||
| Mind | Dis | Vis | ORPO | GRPO | Score | Step SR | Score | Step SR | Score | Step SR | Score | Step SR | |
| Qwen2.5-Coder-32B-Instruct | ✗ | ✗ | ✗ | ✗ | ✗ | 28.7 | 14.6 | 30.1 | 15.5 | 29.8 | 16.1 | 26.2 | 27.9 |
| + Mind | ✓ | ✗ | ✗ | ✗ | ✗ | 55.1 | 36.3 | 55.1 | 36.3 | 57.5 | 38.8 | 52.8 | 33.8 |
| + Dis | ✗ | ✓ | ✗ | ✗ | ✗ | 60.7 | 42.0 | 59.8 | 41.9 | 64.8 | 45.6 | 57.7 | 38.6 |
| + Vis | ✗ | ✗ | ✓ | ✗ | ✗ | 39.6 | 21.5 | 39.3 | 21.2 | 40.9 | 22.5 | 38.6 | 20.8 |
| +Dis + Vis | ✗ | ✓ | ✓ | ✗ | ✗ | 60.5 | 47.6 | 58.2 | 45.2 | 65.5 | 53.1 | 57.8 | 45.0 |
Table 4: Component-wise ablation study on the Mind2Web development set. We incrementally validate the effectiveness of our data construction strategies (Mind vs. Dis vs. Vis) and training stages (SFT ORPO GRPO). Mind : Original training set; Dis : Discriminative Trajectory Mining data; Vis : Synthetic Visual-Structural Grounding data. Note that Triton-GRPO-32B combines all strategies to achieve optimal performance.
Figure 5: Hyperparameter sensitivity. Step Success Rate plotted against negative pool size ( ) and rejection volume ( ). The distinct peak at (Red line) indicates an optimal trade-off, demonstrating that performance degrades under both sparse discrimination signals ( ) and excessive noise ( ).
Figure 6: Scaling analysis of synthetic data. We measure performance across varying data sizes (0–29k). The observed monotonic growth, particularly the +3.3% boost in Cross-Domain (Blue), confirms that exposing the agent to diverse synthetic layouts serves as an effective structural regularizer for unseen websites.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Visualizing Hard Negatives in a flight search list. The model must discern the correct button based on the associated airline context, despite high structural similarity.
Figure 8: Visualization of Counterfactual Rejection. While baseline models often force an action on disabled elements due to instruction bias, Triton correctly outputs a rejection token.
Figure 9: Prompt templates used in the Dual-Agent Consensus pipeline. The Generator ( ) is conditioned on the target to create diverse intents, while the Verifier ( ) is blind to the target to ensure rigorous quality control.
Figure 10: The flattened input representation. We inject explicit IDs (e.g., id="42" ) into the HTML to enable precise referencing. The History block enables the model to maintain context across long horizons.
| Hyperparameter | Value |
| General Configuration | |
| Base Model | Qwen2.5-Coder-32B-Instruct |
| Precision | bfloat16 |
| Optimizer | AdamW |
| Weight Decay | 0.01 |
| LR Scheduler | Cosine |
Table 5: Hyperparameter Settings. We list the specific configurations for SFT, ORPO, and GRPO stages. Note that the learning rate decays according to a cosine schedule across all stages.
| Component | Value | Condition / Description |
| Format Reward ( ) | Awarded if the output strictly follows the JSON/ChatML structure (valid Element ID and Operation fields). | |
| Option Gate ( ) | Acts as a multiplier. if the predicted Element ID matches the ground truth, otherwise. | |
| Element Reward ( ) | Major reward for correctly identifying the target DOM element. | |
| Arg F1 Score ( ) | Token-level F1 score between predicted typed text (e.g., "iphone 13") and ground truth. Only calculated if . | |
| Perfect Bonus ( ) | Extra bonus if both Element and Operation arguments are perfectly correct ( ). | |
| Max Total Reward | 3.1 |
Table 6: Hierarchical Reward Breakdown. The reward function is designed as a cascade: the agent must first satisfy the format, then the element selection, and finally the operation arguments. If the element is wrong ( ), subsequent rewards are zeroed out to prevent reward hacking.