When Adaptive Rewards Hurt: Causal Probing and the Switching-Stability Dilemma in LLM-Guided LEO Satellite Scheduling

In this paper, we propose a dueling double deep Q-network (DDQN)-based adaptive multi-objective handover framework for low Earth orbit (LEO) satellite networks. The proposed method enables dynamic trade-off learning among throughput, blocking probability, and switching cost under time-varying network conditions. Simulation results demonstrate that the proposed approach consistently outperforms conventional baselines, achieving up to 10.3% throughput improvement and near-zero blocking under typical operating conditions.

Large Language Models (LLMs) achieve remarkable reasoning capabilities through reinforcement learning (RL) post-training. However, existing RL post-training commonly relies on uniform data sampling, which ignores the semantic structure of the training data and the evolving capability of the training policy. To address these limitations, we propose Adaptive Data Scheduling (ADS), a dual-level data scheduling framework that combines adaptive sampling across semantic clusters with policy-boundary sample selection. At the cluster level, ADS organizes samples according to semantic patterns and adapts the inter-cluster sampling distribution to prioritize clusters within the policy's current reach. At the sample level, ADS tracks prompts near the policy's capability boundary within each cluster to obtain informative relative advantages. Experimental results across three LLMs and seven reasoning benchmarks demonstrate that ADS improves average accuracy by 5.5% over Group Relative Policy Optimization (GRPO). Notably, ADS consistently improves RL methods with different objective designs, highlighting its potential as a general data scheduling strategy. The source code is available at: https://github.com/Richard-zrx/ADS.

For sparse, structured reinforcement-learning tasks with semantic reward-function interfaces, LLM-generated reward shaping is better framed as debugging than one-shot generation. We study PPO-trained agents using MiniGrid as core evaluation and MuJoCo reaching/locomotion as boundary stress tests. Our qualitative audit finds two recurring one-shot failure modes -- reward flooding and semantic/API misunderstanding -- and a rarer, less reliably distinguished weak-shaping case. We propose diagnostic-driven iterative refinement, using training diagnostics and a failure-mode taxonomy to guide reward-function revision. On sparse structured tasks with diagnosable reward failures, refinement improves DoorKey-8x8 from 2.3% success without shaping to 97.6%, and KeyCorridor from 31.2% one-shot to 86.7% with high seed-to-seed variance. Controls show these gains are not from generic retrying or extra training alone: metrics-only re-prompting causes large drops, while a static-vocabulary control recovers much of the gap. Thus taxonomy prompting is a major mechanism, with only partially isolated incremental evidence for dynamic labels. In prompt-matched comparisons across Haiku 4.5, Opus 5, and DeepSeek V4 Pro, effects depend on task and model; none of the six contrasts survives family-wise correction. A four-call, episode-budget-matched population control is also competitive. Component-removal and sensitivity tests reveal calibration limits. In continuous control, success-based diagnostics can misfire in dense-reward locomotion; return-trend feedback removes one false-positive mechanism without robust locomotion gains. The method is bounded to sparse structured tasks with reliable structured interfaces under PPO training.