cs.AIAug 18, 2026

SignalReasoner: Assessing the Upper Bound of 3B Models for Signal Mathematical Reasoning

Authors: Guozheng Sun

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Post-training with supervised chain-of-thought fine-tuning and reinforcement learning from verifiable rewards has substantially improved the mathematical reasoning capabilities of large language models (LLMs). However, their application to signal processing problems remains relatively under-explored. This report investigates reinforcement fine-tuning strategies for adapting Qwen2.5-3B-Base to graduate-level signal mathematical problems from WirelessMATHBench-XL, a comprehensive benchmark for mathematical reasoning in this domain. We examine two training paradigms: (i) direct reinforcement learning (RL) on WirelessMATHBench-XL with verifiable rewards; and (ii) supervised fine-tuning (SFT) on a distilled wireless-domain chain-of-thought corpus, followed by the same domain-specific RL stage. Across both paradigms, we benchmark Group Relative Policy Optimization (GRPO), Group Sequence Policy Optimization (GSPO), and Geometric-Mean Policy Optimization (GMPO). We aim to assess whether domain-aware CoT SFT serves as an effective initialization for subsequent RL, and whether GSPO or GMPO offer advantages in stability or accuracy over GRPO for signal reasoning tasks. Our best model achieves an overall accuracy of 39.12%, representing a more than threefold improvement over the untrained Base model (12.37%).

Figures & tables

Figure 1: Teaser comparison on WirelessMATHBench-XL. Blue bars show reference strong-model results, while orange/red bars summarize our Qwen2.5-3B experiments. Route A denotes direct RL, and Route B denotes SFT+RL.
Figure 2: Comparison of PPO and GRPO architectures, adapted from DeepSeek-R1 [ 7 ] . PPO requires a separate critic network for advantage estimation via GAE, while GRPO eliminates the critic by computing group-relative advantages from multiple sampled responses.
Figure 3: Distribution of the top 20 key techniques across the 970 source papers in WirelessMATHBench-XL [ 10 ] .
Figure 4: Two-route training design. Route A applies RL directly to Qwen2.5-3B-Base. Route B first performs SFT on Wireless-CoT-Mix and then applies signal-domain RL.
IDRouteInitializationRL Algorithm
Base/Base/
A-GRPORLBaseGRPO
A-GSPORLBaseGSPO
A-GMPORLBaseGMPO
Base-SFTSFTBase + SFT/
B-GRPOSFT + RLBase + SFTGRPO
Table 1: Main experimental methods.
ModelSizeMCQFill-inFECOverallOutput Tokens
SignalReasoner 3B Models
Base3B28.579.039.4212.37907.41
Base-SFT3B42.1126.6824.6128.75604.45
A-GRPO3B57.8930.8827.2334.50668.33
A-GSPO3B61.6530.8826.1834.88279.25
A-GMPO3B51.8832.9826.7034.63395.84
Table 2: Performance of SignalReasoner models on the WirelessMATHBench-XL test set (%). MCQ: Multiple Choice Questions, Fill-in: Fill-in-the-blank, FEC: Full Equation Completion.
Figure 5: SFT loss curves on Wireless-CoT-Mix. The blue curve shows smoothed training loss, and the red curve shows validation loss evaluated every 20 steps.
Figure 6: Algorithm comparisons within each route. Top row: direct RL from the Base model (Route A), truncated at 2,000 steps. Bottom row: the SFT+RL route, where RL starts after supervised training on Wireless-CoT-Mix (Route B), truncated at 1,400 steps.
Figure 7: Route comparisons per algorithm. Solid lines denote Route A, direct RL from the Base model, truncated at 2,000 steps. Dashed lines denote Route B, the SFT+RL route, truncated at 1,400 steps.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

SFT
Base modelQwen2.5-3B-Base
SFT datasetWireless-CoT-Mix (3,542 examples)
Max response length768
Learning rate3e-6
Batch size36
Epochs1
Appendix
Table A1 : Detailed training hyperparameters.
ModelSizeMCQFill-inFECOverallOutput Tokens
Proprietary Models
GPT-5–63.9163.2041.3657.87–
GPT-5-mini–67.6753.9940.3153.00–
GPT-5-nano–57.1437.8230.3739.25–
GPT-4o–54.1443.6224.6140.37–
o4-mini–67.6749.5640.3150.38–
Appendix
Table A2 : Extended comparison on the WirelessMATHBench-XL test set (%).
More in this topic

LLM Fine-Tuning

LLM fine-tuning adapts pretrained large language models to tasks, domains, languages, and behaviors through additional training.

See all 755 papers

Explore similar work

CardsList

  1. Learn Hard Problems During RL with Reference Guided Fine-tuning

    Mar 1, 2026Yangzhen Wu, Shanda Li, Zixin Wen +5LLM Mathematical ReasoningRL for Language Model Reasoning

    UC Berkeley · Carnegie Mellon University · ByteDance Seed

  2. Offline Exploration-Aware Fine-Tuning for Long-Chain Mathematical Reasoning

    Mar 17, 2026Yongyu Mu, Jiali Zeng, Fandong Meng +2LLM Mathematical ReasoningSupervised Fine-Tuning

    NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China · Pattern Recognition Center, WeChat AI, Tencent Inc, China

  3. Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning

    Apr 23, 2026Yongcan Yu, Lingxiao He, Jian Liang +5Reinforcement LearningRL for Language Model Reasoning

    1NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · *Work done during an internship at Meituan · School of Artificial Intelligence, University of Chinese Academy of Sciences +2