cs.LGApr 5, 2026
SaveSubspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
Organizations: OPTML, Michigan State University · MIT-IBM Watson AI Lab, IBM Research · University of Minnesota · University of Michigan ·
Abstract
Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, and task-specific requirements, leading to "constrained" optimization problems for model steering and adaptation. However, solving such problems remains largely underexplored and is particularly challenging due to interference between the primary objective and constraint objectives during optimization. In this paper, we propose a subspace control framework for constrained model training. Specifically, (i) we first analyze, from a model merging perspective, how spectral cross-task interference arises and show that it can be resolved via a one-shot solution that orthogonalizes the merged subspace; (ii) we establish a connection between this solution and gradient orthogonalization in the spectral optimizer Muon; and (iii) building on these insights, we introduce SIFT (spectral interference-free training), which leverages a localization scheme to selectively intervene during optimization, enabling controllable updates that mitigate objective-constraint conflicts. We evaluate SIFT across four representative applications: (a) machine unlearning, (b) safety alignment, (c) text-to-speech adaptation, and (d) hallucination mitigation. Compared to both control-based and control-free baselines, SIFT consistently achieves substantial and robust performance improvements across all tasks. Code is available at https://github.com/OPTML-Group/SIFT.
Figures & tables
Figure 1: Schematic overview of proposed subspace control framework, SIFT. ( A ) Performance across four model steering tasks (detailed in Tables 1 and 2 ), compared with the baseline BLUR ( Reisizadeh et al., 2025 ) . ( B ) When and where to control : SIFT enables selective intervention at targeted layers and training steps ( i.e. , spatial-temporal localization). ( C ) How to control : Built on spectral optimizer Muon, SIFT leverages gradient orthogonalization (the matrix sign function) to mitigate subspace interference.
| Application | Primary Objective | Constraint Objective |
| Machine unlearning (§ 5.2 ) | MSE (mean squared error) loss of representation alignment on a retain dataset ( Li et al., 2024 ) | RMU (representation misdirection unlearning) loss on a forget dataset ( Li et al., 2024 ) |
| Safety alignment (§ 5.3 ) | CE (cross-entropy)-based SFT (supervised fine-tuning) loss on a utility dataset | DPO (direct preference optimization) loss on a safety dataset ( Rafailov et al., 2023 ) |
| Text-to-speech adaptation (§ 5.4 ) | CE loss on text generation ( Zeng et al., 2024b ) | CE loss on speech generation ( Zeng et al., 2024b ) |
| Hallucination mitigation (§ 5.5 ) | CE-based prediction loss on non-hallucinated tokens | Negative CE loss on hallucinated tokens |
Table 1: Specification of the constrained formulation across different applications.
Figure 2: Visualization of cosine similarity across optimization steps ( temporal dimension) and model layers ( spatial dimension) in LLM unlearning. The top and right marginal plots summarize the counts of across steps and layers, respectively. Red stars mark steps and layers need for control.
Figure 3: Utility loss under different descent directions starting from a model at step 35 of the unlearning process. Multiple update steps are performed along the full gradient, projected gradient, and removed component, respectively. The unlearning setup is the same as in Fig. 2 .
Figure 4: Sparse localization across applications. Temporal sparsity is the fraction of all training steps, and spatial sparsity is the fraction of all model components where SIFT is activated.
| Applications | Training Datasets | Base LLM | Evaluation |
| Machine unlearning | Forget: WMDP ( Li et al., 2024 ) Retain: Wikitext ( Merity et al., 2016 ) | zephyr-7b-beta | Unlearning ( ): ES-Bio/Cyber ( Yuan et al., 2024 ) , MCQ-Bio/Cyber ( Li et al., 2024 ) Utility ( ): MMLU ( Hendrycks et al., 2021 ) , TruthfulQA ( Lin et al., 2021 ) , IFEval ( Zhou et al., 2023 ) , GSM8K ( Cobbe et al., 2021 ) |
| Safety alignment | Safety: PKU-SafeRLHF ( Ji et al., 2024 ) Utility: Alpaca ( Taori et al., 2023 ) | Llama-2-7b | Safety ( ) ( Wang et al., 2025 ) : Strong Reject, JBB-Behaviors, Wild Jailbreak Utility ( ): MMLU, GSM8K, IFEval, MNLI ( Wang et al., 2019 ) , MRPC ( Wang et al., 2019 ) |
| Hallucination mitigation | RAGTruth ( Niu et al., 2024 ) | Llama-2-7B-Chat | Hallucination rate ( ) ( Niu et al., 2024 ) Utility ( ): MMLU, GSM8K, TruthfulQA, QNLI |
| Text-to-speech adaptation | ESNLI ( Camburu et al., 2018 ) COSE ( Rajani et al., 2019 ) OpenBookQA ( Mihaylov et al., 2018 ) | GLM-4-Voice ( Zeng et al., 2024a ) | Test accuracy ( ) on ESNLI, COSE, and OpenBookQA under four cross-modal input -output settings: “Audio Text”, “Audio Audio”, “Text Audio”, “Text Text” |
Table 2: Overview of experimental setups for different applications.
| Methods | Unlearning Performance (%) | Utility Performance (%) | Runtime (min) | ||||||
| ES-Bio | ES-Cyber | MCQ-Bio | MCQ-Cyber | MMLU | TruthfulQA | IFEval | GSM8K | ||
| Base Model | 62.3 | 46.7 | 64.1 | 44.8 | 58.2 | 39.5 | 10.4 | 35.6 | N/A |
| AdamW | 14.6 | 26.3 | 31.7 | 29.4 | 57.1 | 40.8 | 9.2 | 33.9 | 8.8 |
| Muon | 14.2 | 30.5 | 31.3 | 28.6 | 56.4 | 37.9 | 8.7 | 34.1 | 11.8 |
| POME | 15.8 | 29.6 | 29.2 | 27.5 | 57.3 | 38.4 | 9.6 | 34.7 | 9.9 |
| BLUR | 9.3 | 24.8 | 28.6 | 27.3 | 57.1 | 39.2 | 9.1 | 33.5 | 9.4 |
Table 3: Performance of LLM unlearning for removing hazardous content generation on WMDP. ES-Bio/Cyber denotes the entailment score (ES) ( Yuan et al., 2024 ; Fan et al., 2025 ) measuring unlearning effectiveness via factual consistency with the pre-unlearned model on WMDP Bio and Cyber sets. MCQ-Bio/Cyber denotes multi-choice question (MCQ) accuracy on the same sets; Model utility is evaluated on standard benchmarks. Superscripts indicate standard deviations over 10 random trials.
Figure 5: Sensitivity analysis of SIFT with respect to the top- subspace dimension. Left: effective rank of momentum matrices. Right: unlearning and utility performance under varying .
| Methods | Safety Performance (%) | Utility Performance (%) | Runtime (min) | ||||||
| SR | JBB | WJ | MMLU | GSM8K | IFEval | MNLI | MRPC | ||
| Base model | 19.5 | 14.0 | 47.6 | 41.3 | 14.7 | 34.2 | 42.8 | 69.5 | N/A |
| AdamW | 33.2 | 28.0 | 52.8 | 36.6 | 8.4 | 31.9 | 36.2 | 62.7 | 110.3 |
| Muon | 36.4 | 26.0 | 52.0 | 36.1 | 9.5 | 30.4 | 32.8 | 63.3 | 191.2 |
| POME | 36.7 | 26.0 | 51.6 | 38.7 | 9.2 | 31.6 | 32.5 | 62.1 | 115.4 |
| BLUR | 35.8 | 29.0 | 54.4 | 37.4 | 9.8 | 31.3 | 37.6 | 65.9 | 114.7 |
Table 4: Safety alignment performance across optimization methods. Higher scores on Strong Reject, JBB, and Wild Jailbreak indicate stronger safety, while higher scores on downstream tasks (MMLU, GSM8K, IFEval, MNLI, MRPC) indicate better utility. Evaluation metrics are specified in Table 2 and results format follows Table 3 .
| Methods | Audio → Text ( ) | Audio → Audio ( ) | Text → Text ( ) | Text → Audio ( ) | Runtime (min) | ||||||||
| ESNLI | COSE | OpenBook | ESNLI | COSE | OpenBook | ESNLI | COSE | OpenBook | ESNLI | COSE | OpenBook | ||
| Base | 27.2 | 42.1 | 12.6 | 20.1 | 41.6 | 11.2 | 60.6 | 66.2 | 54.7 | 42.8 | 34.5 | 43.4 | N/A |
| AdamW | 70.3 | 46.7 | 48.6 | 68.6 | 48.1 | 27.5 | 78.9 | 74.1 | 68.2 | 74.1 | 52.9 | 54.2 | 44.6 |
| Muon | 73.6 | 48.5 | 53.4 | 71.5 | 49.1 | 27.3 | 79.9 | 72.8 | 64.4 | 74.3 | 52.4 | 53.8 | 47.9 |
| POME | 73.3 | 47.2 | 53.4 | 70.6 | 48.2 | 25.7 | 80.6 | 72.3 | 64.5 | 74.1 | 52.2 | 53.7 | 24.7 |
| BLUR | 69.6 | 43.2 | 47.1 | 68.6 | 44.1 | 20.4 | 80.1 | 72.2 | 63.1 | 77.4 | 53.6 | 51.5 | 46.1 |
Table 5: Test accuracy ( ) of the speech LLM trained on interleaved speech–text data constructed from ESNLI, COSE, and OpenBookQA under four input–output settings (e.g., “Audio → Text” denotes audio input and text output). Evaluation metrics and result format follow Tables 2 and 3, respectively.
Figure 6: Localization patterns under the dataset ESNLI. Components are grouped by Transformer layer: each layer comprises self-attention module (Q, K, V, Out) and feed-forward module (FC1, FC2). The presenting format follows Fig. 2 .
| Methods | Hallucination Rate ( ) | Utility Performance ( ) | Runtime (min) | ||||
| MMLU | GSM8K | TruthfulQA | QNLI | MNLI | |||
| Base model | 73.2 | 46.5 | 20.4 | 30.2 | 68.7 | 56.2 | N/A |
| AdamW | 39.1 | 46.1 | 17.4 | 28.9 | 68.3 | 56.2 | 8.1 |
| Muon | 38.5 | 46.2 | 17.8 | 29.0 | 68.2 | 56.2 | 12.0 |
| POME | 41.1 | 46.0 | 17.8 | 29.2 | 68.2 | 56.1 | 10.6 |
| BLUR | 44.3 | 46.0 | 15.4 | 27.1 | 68.0 | 56.2 | 8.6 |
Table 6: Performance of hallucination mitigation on RAGTruth. Hallucination rate measures the proportion of responses containing hallucinated content (judged by GPT-5.2); lower is better.
Figure A1: Localization patterns of SIFT for safety alignment, similar to Fig. 2 .
Figure A2: Sensitivity analysis on subspace dimension in SIFT for safety alignment. ( Left ) The effective rank of the momentum matrices computed across SIFT-localized layers (15, 18, and 22) under different energy thresholds. ( Right ) Performance of SIFT under varying , where Safety denotes the average over all safety metrics, and Utility denotes the average over all utility metrics in Table 4 .
Figure A3: Example of interleaved speech–text data constructed from COSE’s training set.
| Input query | Summarize the following news within 161 words: …Five people were infected and three died in the past year in Kansas from listeria that might be linked to Blue Bell Creameries products, according to the CDC… |
| Original Model | … This is the third time Blue Bell has taken action due to listeria contamination, and the company is cooperating with investigations. No illnesses have been reported directly linked to the contaminated ice cream, but five people in Kansas have died from listeriosis in the past year after consuming Blue Bell products. |
| Mitigated model | …This recall follows past listeria outbreaks in Kansas and Texas, where five people were infected and three died, some after consuming milkshakes made with Blue Bell ice cream. Blue Bell is cooperating with authorities, emphasizing safety, and other Blue Bell products are not affected. |
Table A1: Example responses from the base model and the SIFT-updated model for an input query sampled from the RAGTruth dataset. Red text indicates hallucinated content.
Figure A4: In each Transformer layer, “Q”, “K”, and “V” denote the query, key, and value projection matrices, and “O” denotes the output projection of the self - attention module. “G”, “U”, and “D” denote the gating, up - projection, and down - projection components of the feed - forward network.