Cognitively Layered Data Synthesis for Domain Adaptation of LLMs to Space Situational Awareness
Authors: Ding Linghu, Cheng Wang, Da Fan, Wei Shi, Kaifeng Yin, Xiaoliang Xue, Fan Yang, Haiyi Ren, Cong Zhang
Organizations: Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology, Beijing, China · China Academy of Space Technology, Beijing, China · State Key Laboratory of Space Information System and Integrated Application, Beijing, China
Large language models (LLMs) demonstrate exceptional performance on general-purpose tasks. however, transferring them to complex engineering domains such as space situational awareness (SSA) remains challenging owing to insufficient structural alignment with mission chains, the absence of higher-order cognitive supervision, and poor correspondence between data quality criteria and engineering specifications. The core bottleneck is the construction of high-quality supervised fine-tuning (SFT) datasets. To this end, we propose BD-FDG (Bloom's Taxonomy-based Domain-specific Fine-tuning Data Generation), a framework that addresses incomplete knowledge coverage, shallow cognitive depth, and limited quality controllability through three mechanisms: structured knowledge organization, cognitively layered question modeling, and automated quality control. The framework uses a knowledge tree to ensure structured corpus coverage, designs a question generation scheme spanning nine categories and six cognitive levels from Remember to Create to produce samples with a continuous difficulty gradient, and applies a multidimensional scoring pipeline to enforce domain rigor and consistency. Using BD-FDG, we construct SSA-SFT, a domain dataset of approximately 230K samples, and fine-tune Qwen3-8B to obtain SSA-LLM-8B. Experiments show that SSA-LLM-8B achieves relative BLEU-1 improvements of 144% (no-think) and 176% (think) on the domain test set and a win rate of 82.21% over the baseline in arena comparisons, while largely preserving general benchmark performance (MMLU-Pro, MATH-500). These results validate SFT data construction driven by cognitive layering as an effective paradigm for complex engineering domains and provide a transferable framework for domain-specific LLM adaptation.
Figures & tables
Figure 1: Architecture of the BD-FDG framework. The domain knowledge base construction stage (left) parses mission chain literature into structured chunks and builds a hybrid dense–sparse retrieval index for multisource context assembly. The question generation stage (middle) applies nine Bloom’s Taxonomy-guided prompt templates across six cognitive levels to produce questions with graduated difficulty from retrieved context. The SFT data synthesis stage (right) distills reasoning traces and answers from teacher models, scores each sample along four quality dimensions, and filters by a composite threshold to yield the final SSA-SFT dataset.
No.
Type Name
Cognitive Level
Assessment Objective
Q1
Concept Discrimination
Remember/Understand
Understanding and distinguishing core concepts
Q2
Principle Explanation
Understand
In-depth understanding and explanation of technical principles
Q3
Formula Derivation
Understand/Apply
Mathematical modeling and theoretical analysis
Q4
Parameter Calculation
Apply
Design and calculation of key parameters
Q5
Algorithm Implementation
Apply/Analyze
Algorithm design and optimization
Q6
Performance Analysis
Analyze
System performance analysis and comparison
Table 1: Question type system guided by Bloom’s Taxonomy (Q1–Q9) for the space situational awareness domain.
Figure 2: Overview of the SSA-SFT and SSA-Test datasets. Left: three-tier mission chain knowledge system. Right: distribution of question types and sample counts.
Dataset
Task Type
no-think
think
Qwen3-8B
SSA-LLM-8B
Qwen3-8B
SSA-LLM-8B
SSA-Test
Knowledge
21.33
52.08
20.75
57.23
AIME24
Math
33.33
23.33
66.67
66.67
AIME25
Math
16.67
23.33
60.00
56.67
C-Eval
Exam
79.35
78.01
83.28
79.20
GPQA Diamond
Knowledge
51.52
44.44
60.10
60.10
Table 2: Test results of SSA-LLM-8B and Qwen3-8B on various test sets. All values are in %. Bold indicates the better or tied best value under the same mode.
Inference Mode
Win Rate (%)
Lower CI (%)
Upper CI (%)
no-think
82.21
81.50
82.91
think
73.54
72.84
74.44
Table 3: Arena Battle results on the SSA-Test dataset (SSA-LLM-8B vs. Qwen3-8B).
Metric
no-think
think
Qwen3-8B
SSA-LLM-8B
Qwen3-8B
SSA-LLM-8B
BLEU-1
21.33
52.08
20.75
57.23
BLEU-2
12.40
35.26
12.81
39.53
BLEU-3
7.80
25.70
8.54
29.18
BLEU-4
4.97
19.26
5.77
22.16
ROUGE-1-F
42.31
53.03
45.30
54.62
Table 4: Fine-grained metrics on the SSA-Test dataset. All values are in %. Bold indicates the better or tied best value under the same mode.
Figure 3: Two-dimensional ablation results. The central heatmap shows average quality scores for each (α,K) configuration; marginal bar plots depict the main effects of Top- K (top) and α (right). The optimal configuration (α=0.50,K=5) attains a score of 7.55.