cs.AIMar 7, 2026
SaveVisualDeltas: Learning Preferences from Visual Quality Perturbations
Organizations: Peking University, Beijing, China · Alibaba Group, Hangzhou, China · Zhejiang University, Hangzhou, China
Abstract
We present VisualDeltas, a lightweight preference-learning framework that extracts supervision from visual quality variations in multimodal data. By leveraging the systematic impact of image quality on visual perception and reasoning, VisualDeltas induces informative preference signals without relying on human annotations or external teachers. The framework supports both label-free and label-based regimes, enabling flexible use of available supervision when present. Across diverse multimodal benchmarks and model scales, VisualDeltas consistently outperforms rejection-sampling fine-tuning and improves generalization, and extends naturally to a range of visual degradations.
Figures & tables
Figure 1: DPO pair construction via multimodal input quality perturbation. HQ and LQ inputs for the same multimodal QA task induce correct and incorrect reasoning, forming a natural preference pair.
Figure 2: Overview of VisualDeltas. For each user query, the model generates paired responses under both HQ and LQ image views. We compare with SFT (supervised fine-tuning on HQ-correct responses only) and two VisualDeltas variants: VD-LF (label-free) uses all HQ vs. LQ pairs without correctness filtering, while VD-LB (label-based) selects only HQ-correct vs. LQ-wrong pairs. Both VisualDeltas variants apply DPO exclusively with HQ context during training.
| Model | Method | Datasets | ||||
|---|---|---|---|---|---|---|
| HiTab | WikiTQ | VQA | GQA | MathVision | ||
| Qwen2.5-7B-VL | ||||||
| Inference Only (No Training) | ||||||
| Baseline | 67.93 | 66.40 | 60.00 | 39.95 | 24.87 | |
| Train Set: HiTab | ||||||
| SFT (HQ Correct Only) | 69.89 (+1.96) | 65.90 (-0.50) | 60.85 (+0.85) | 39.05 (-0.90) | 22.76 (-2.11) | |
Table 1: Comparison of Qwen2.5-7B-VL and Qwen2.5-3B-VL under different training strategies, grouped by training dataset. Numbers are absolute accuracy (%), with differences relative to the Vanilla inference-only baseline shown in parentheses. SFT fine-tunes on HQ correct responses (using rejection-sampled model-generated reasonings), VD-LF constructs label-free HQ vs. LQ preferences, and VD-LB applies label-based VisualDeltas using HQ-correct vs. LQ-wrong pairs. Row and cell coloring: gray marks dataset groupings, blue marks VisualDelta-based results, dark green highlights best results, light green indicates positive gains, light red indicates negative changes, and dark red marks the largest drop.
| Method | Test on LQ (0.1 Res) | ||||
|---|---|---|---|---|---|
| HiTab | WikiTQ | VQA | GQA | MathVision | |
| Qwen2.5-7B-VL | |||||
| No Training | 33.46 | 40.00 | 50.55 | 33.40 | 23.95 |
| Train: HiTab | |||||
| SFT | 34.34 | 45.35 | 51.20 | 32.30 | 18.55 |
| VD-LF | 36.43 | 47.95 | 52.55 | 36.80 | 22.37 |
Table 2: Generalization to low-quality image inputs (Qwen2.5-7B-VL). All models are trained on HQ images and tested on LQ images. SFT fine-tunes on HQ correct responses, VD-LF constructs label-free HQ vs. LQ preferences, and VD-LB applies label-based filtering using HQ-correct vs. LQ-wrong pairs. Best results per column are highlighted in bold with dark green background. Row and cell coloring: gray marks dataset groupings, blue marks VisualDeltas results, and dark green indicates best performance.
Figure 3: Sample category distribution on HiTab. Quality-Sensitive samples (HQ correct, LQ wrong) comprise 38.3% of the dataset, providing clean HQ LQ preference pairs for VD-LB training.
Figure 4: Response length distribution by category. LQ responses are consistently longer than HQ responses, suggesting that degraded visual inputs trigger compensatory but ineffective reasoning. After DPO training on VisualDeltas preference pairs, the HQ distribution shifts left and becomes sharper, with reduced mean and median token lengths. This demonstrates that DPO improves reasoning efficiency—models learn to produce more concise responses while maintaining higher accuracy.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Visual Modality | Task Type | Train Size | Test Size |
|---|---|---|---|---|
| HiTab | Table images | Hierarchical table QA | 7,417 | 1,584 |
| WikiTableQuestions | Table images | Table QA | 8,000 | 2,000 |
| VQA v2 | Natural images | Open-ended VQA | 8,000 | 2,000 |
| GQA | Natural images | Compositional VQA | 8,000 | 2,000 |
| MathVision | Mixed visual formats | Visual math reasoning | 8,000 | 2,000 |
Table 3: Summary of datasets used in this work. Training is conducted on HiTab, VQA, and GQA, while evaluation is performed on all datasets. For HiTab and WikiTableQuestions, tables are converted from JSON/CSV format to images.
Figure 5: Visual Examples from Different Datasets. (a)-(b) Table datasets (HiTab, WikiTableQuestions) feature complex, dense structures requiring precise visual grounding. (c)-(d) Natural image datasets (VQA, GQA) contain real-world scenes with varying complexity. (e) MathVision contains simple mathematical expressions and diagrams that rely more on reasoning than visual details. This visual complexity hierarchy explains why HiTab/WTQ show strong resolution dependence while MathVision remains resolution-independent.
Figure 6: Accuracy vs. Resolution Across Datasets. Performance trends across different datasets under varying image resolutions. HiTab and WTQ show strong resolution dependence (accuracy doubling from 10% to 100% resolution), reflecting their visual complexity with dense, structurally intricate tables. MathVision maintains nearly flat performance across all resolutions, indicating reliance on reasoning rather than visual details. VQA and GQA show intermediate resolution sensitivity. This demonstrates that VisualDeltas specifically targets visually grounded reasoning tasks.
Figure 7: Resolution-Accuracy Relationship. The 7B model exhibits a characteristic three-phase pattern: stability at high resolution (100%–40%), rapid degradation in the critical zone (40%–20%), and severe failure at extreme low resolution (20%–10%). The 3B model shows a similar pattern with systematically lower accuracy, confirming the robustness of this phenomenon across model sizes.
| Resolution | Accuracy | Correct | Total | Change |
|---|---|---|---|---|
| 100% (1.0) | 67.11% | 1063 | 1584 | – |
| 80% (0.8) | 67.99% | 1077 | 1584 | +0.88% |
| 60% (0.6) | 67.99% | 1077 | 1584 | +0.88% |
| 40% (0.4) | 66.60% | 1055 | 1584 | -0.51% |
| 20% (0.2) | 56.50% | 895 | 1584 | -10.61% |
| 10% (0.1) | 33.40% | 529 | 1584 | -33.71% |
Table 4: Accuracy metrics across resolution levels for Qwen2.5-VL-7B.
Figure 8: Sample Stability Heatmap. Failure transitions are unidirectional—once a question begins failing, it rarely recovers.
| Train Set | VD-S Total (7B) | HQ vs. HQ Total (7B) | VD-S Total (3B) | HQ vs. HQ Total (3B) |
|---|---|---|---|---|
| HiTab | +10.23 | +10.21 | +10.10 | +9.42 |
| VQA | +10.39 | +8.34 | +9.85 | +8.50 |
| GQA | +12.84 | +9.12 | +10.95 | +8.91 |
Table 5: Cumulative Accuracy Gains: VD-LB vs. HQ-only DPO across all evaluation benchmarks. VD-LB achieves higher total gains than correctness-based DPO alone, demonstrating that visual quality deltas provide complementary supervision beyond mere correctness.
| Train | Method | Qwen2.5-7B-VL | Qwen2.5-3B-VL | ||||||
|---|---|---|---|---|---|---|---|---|---|
| HiTab | WikiTQ | VQA | GQA | HiTab | WikiTQ | VQA | GQA | ||
| HiTab | VD-S | 71.40 (+3.47) | 69.90 (+3.50) | 63.55 (+3.55) | 42.90 (+2.95) | 61.11 (+2.15) | 51.35 (+12.90) | 60.85 (+4.55) | 39.75 (+3.45) |
| HQ vs. HQ | 71.15 (+3.22) | 68.95 (+2.55) | 63.15 (+3.15) | 40.90 (+0.95) | 60.67 (+1.71) | 50.05 (+11.60) | 61.50 (+5.20) | 39.80 (+3.50) | |
| VQA | VD-S | 70.58 (+2.65) | 68.15 (+1.75) | 68.20 (+8.20) | 44.75 (+4.80) | 59.09 (+0.13) | 52.45 (+14.00) | 61.45 (+5.15) | 42.50 (+6.20) |
| HQ vs. HQ | 70.20 (+2.27) | 68.60 (+2.20) | 66.55 (+6.55) | 43.65 (+3.70) | 61.24 (+2.28) | 50.60 (+12.15) | 61.90 (+5.60) | 41.35 (+5.05) | |
| GQA | VD-S | 69.19 (+1.26) | 69.60 (+3.20) | 66.00 (+6.00) | 47.50 (+7.55) | 61.36 (+2.40) | 50.60 (+12.15) | 58.70 (+2.40) | 42.65 (+6.35) |
Table 6: Comparison between VD-S (HQ Correct vs. LQ Wrong) and DPO (HQ Correct vs. HQ Wrong) across Qwen2.5-7B-VL and 3B-VL. Bold indicates the best score per model for each metric. Numbers in parentheses indicate absolute improvements relative to the respective baseline.
| Train Set | VD-S Total Diff (7B) | HQ vs. HQ Total Diff (7B) | VD-S Total Diff (3B) | HQ vs. HQ Total Diff (3B) |
|---|---|---|---|---|
| HiTab | +10.23 | +10.21 | +10.10 | +9.42 |
| VQA | +10.39 | +8.34 | +9.85 | +8.50 |
| GQA | +12.84 | +9.12 | +10.95 | +8.91 |
Table 7: Summed absolute accuracy improvements across all evaluation benchmarks, for both 7B and 3B models, relative to the baseline.
| Model Size | |||||||||||
| Train Set | DPO Ratio | 7B | 3B | ||||||||
| HiTab | WikiTQ | VQA | GQA | MathVision | HiTab | WikiTQ | VQA | GQA | MathVision | ||
| Train Set: HiTab | |||||||||||
| 1.0 vs 0.1 | 71.91 | 67.15 | 62.65 | 42.05 | 25.16 | 61.62 | 52.15 | 61.55 | 40.65 | 18.78 | |
| 1.0 vs 0.2 | 70.71 | 67.95 | 62.40 | 39.90 | 25.07 | 59.72 | 49.70 | 60.40 | 38.15 | 17.20 | |
| 1.0 vs 0.4 | 70.52 | 66.10 | 60.10 | 36.10 | 24.18 | 59.60 | 47.20 | 59.75 | 39.95 | 16.61 | |
Table 8: Effect of resolution perturbation strength on label-free DPO (VD-LF) across training sets and model sizes. Each DPO ratio compares the original resolution ( ) with a downsampled counterpart. Results are reported in accuracy (%). For each training set, the best performance per task and model size is highlighted in green.
| Method | HiTab | WikiTQ | VQA | GQA | MathVision |
|---|---|---|---|---|---|
| Baseline | 67.93 | 66.40 | 60.00 | 39.95 | 24.87 |
| Gaussian Noise | 72.22 | 67.55 | 63.10 | 43.85 | 25.23 |
| Motion Blur | 72.22 | 68.60 | 64.00 | 42.15 | 24.54 |
| VD-LF (Ours) | 71.91 | 67.15 | 62.65 | 42.05 | 25.16 |
| VD-LB (Ours) | 71.40 | 69.90 | 63.55 | 42.90 | 25.13 |
Table 9: Comparison of noise-based methods vs. VisualDeltas on HiTab training set (Qwen2.5-7B-VL). All values are accuracy (%). Our methods (VD-LF and VD-LB, both using Resolution Reduction) are highlighted in light blue.
| Paradigm | Method | HiTab | Out-of-Domain | |||
|---|---|---|---|---|---|---|
| WikiTQ | VQA | GQA | MathVision | |||
| Baseline | ||||||
| Baseline (no training) | 67.93 | 66.40 | 60.00 | 39.95 | 24.87 | |
| Large-Model-Guided (External Teacher Required) | ||||||
| 7B-guided (7B pref vs. 3B dispref) | 70.01 (+2.08) | 67.25 (+0.85) | 60.90 (+0.90) | 39.45 (-0.50) | 22.66 (-2.21) | |
| 32B-guided (32B pref vs. 7B dispref) | 67.55 (-0.38) | 70.55 (+4.15) | 65.10 (+5.10) | 43.70 (+3.75) | 21.88 (-2.99) | |
Table 10: Detailed comparison of large-model-guided preference learning vs. single-model VisualDeltas. Experiments are conducted on the same 7B target model. Large-model-guided methods require external larger models (32B or 7B) to construct preferences, while VisualDeltas uses only the 7B model itself with resolution-induced deltas. Numbers show absolute accuracy (%) with gains/losses relative to baseline in parentheses. Best results per column are in bold with dark green background. VisualDeltas rows are highlighted in light blue. Light green indicates positive gains, light red indicates negative changes, and dark red marks the largest drop. Notably, VD-LF achieves the best HiTab performance (71.91%), and VisualDeltas methods maintain positive gains across all benchmarks unlike large-model-guided methods which hurt MathVision.
More in this topic
Preference Learning
Preference learning infers people’s values and trade-offs from choices, comparisons, rankings, and feedback to guide predictions and decisions.
See all 150 papers