3DGS-VBench+ and 3DGSV-Assessor: A Multi-Dimensional Subjective Benchmark and a Reference-Flexible Objective Model for 3D Gaussian Splatting Video Quality Assessment
Organizations: School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · Basis Independent Silicon Valley, San Jose, CA 95134, USA · JD.COM, Beijing, 100176, China
Abstract
3D Gaussian Splatting (3DGS) enables real-time novel view synthesis (NVS), but its heavy storage makes compression essential, and both training and compression introduce representation-specific distortions. Since generative compression in many methods treats geometric and color attributes via different parameters, it can induce dimension-specific distortions, motivating multi-dimensional assessment. Moreover, real-world captures generally lack an aligned ground-truth video for a synthesized, continuous camera trajectory, leaving only sparse captured views as references. To enable multi-dimensional quality assessment of 3DGS videos with scarce aligned references, we first present 3DGS-VBench+, a large-scale subjective video quality assessment (VQA) benchmark comprising 3{,}036 3DGS reconstructions, each paired with a rendered video, across 11 real-world and 22 synthetic scenes, generated by 6 representative compression algorithms at systematically designed levels and annotated with 9{,}108 MOSs covering overall, geometry, and color quality. Based on 3DGS-VBench+, we propose 3DGSV-Assessor, a reference-flexible, large multimodal framework that scores all three dimensions in one generation pass and fuses a ranking-based reference cue into the frozen regressor through a bounded gate, supporting both reduced-reference (RR) and no-reference (NR) assessment. Extensive experiments demonstrate state-of-the-art correlation with human perception on 3DGS-VBench+ and strong zero-shot generalization to other NVS benchmarks. The dataset and code will be released at https://github.com/YukeXing/3DGSV-Assessor.
Figures & tables
| Dataset | Name | Year | Scenes | NVS Models | Render | Distortion | Annotation | Number of | Number of | MOSs/JODs | ||
| Type | Syn | Real | NeRF | 3DGS | Mode | Level design | NeRF/3DGS | videos/imgs | ||||
| IQA | NVS-QA [ 20 ] | 2025 | - | 13 | 2 | 3 | 1 viewpoint | ✗ | MOS | 65 | 65 | 65 |
| 3DGS-IEval-15K+ [ 21 ] | 2026 | - | 10 | - | 6 | 20 viewpoints | ✓ | MOS | 760 | 15,200 | 45,600 | |
| [7pt/5pt] VQA | NeRF-QA [ 22 ] | 2023 | 4 | 4 | 7 | - | ✗ | DMOS | 48 | 48 | 48 | |
| NeRF-VSQA [ 23 ] | 2024 | 8 | 8 | 7 | - | +Front | ✗ | DMOS | 88 | 88 | 88 | |
| FFV [ 24 ] | 2024 | - | 22 | 8 | - | Front | ✗ | Pairs | 220 | 220 | 144 | |
| Type | Parameter | CL1 | CL2 | CL3 | CL4 | CL5 |
| Real | g-size | 1 | 2 | |||
| c-size | 1 | 2 | ||||
| reg | 4 1e-6 | 3 1e-6 | 2 1e-6 | 1 1e-6 | 1 1e-7 | |
| Syn | g-size | 1 | ||||
| c-size | 1 | 2 | ||||
| reg | 5 1e-6 | 3 1e-6 | 2 1e-6 | 1 1e-6 | 1 1e-7 |
| Type | Parameter | CL1 | CL2 | CL3 | CL4 | CL5 |
| Real | hashmap | |||||
| (codebook, rvq_depth) | ( , 1) | ( , 1) | ( , 1) | ( , 6) | ( , 6) | |
| lambda | 0.014 | 0.012 | 0.01 | 0.006 | 0.0005 | |
| Syn | hashmap | |||||
| (codebook, rvq_depth) | ( , 1) | ( , 1) | ( , 1) | ( , 6) | ( , 6) | |
| lambda | 0.022 | 0.020 | 0.018 | 0.015 | 0.004 |
| Type | Parameter | CL1 | CL2 | CL3 | CL4 | CL5 |
| Real | prune | 0.97 | 0.95 | 0.90 | 0.85 | 0.66 |
| (c-ratio, c-size) | (1, 2) | (1, ) | (1, ) | (1, ) | (0.6, ) | |
| Syn | prune | 0.95 | 0.90 | 0.85 | 0.75 | 0.66 |
| (c-ratio, c-size) | (1, 1) | (1, 2) | (1, ) | (1, ) | (0.6, ) |
| Type | Parameter | CL1 | CL2 | CL3 | CL4 | CL5 (default) |
| Real | (c-size, c-include) | (1, 0.6) | (2, 0.6) | ( , 0.6) | ( , 0.6) | ( , 0.6 1e-6) |
| (g-size, g-include) | (1, 0.3) | ( , 0.3) | ( , 0.3) | ( , 0.3) | ( , 0.3 1e-5) | |
| Syn | (c-size, c-include) | (1, 0.6) | (2, 0.6) | ( , 0.6) | ( , 0.6) | ( , 0.6 1e-6) |
| (g-size, g-include) | (1, 0.3) | ( , 0.3) | ( , 0.3) | ( , 0.3) | ( , 0.3 1e-5) |
| Param | CL1 | CL2 | CL3 | CL4 | CL5 | CL6 | CL7 | CL8 |
| vsize (real) | 0.350 | 0.250 | 0.200 | 0.160 | 0.120 | 0.080 | 0.050 | 0.001 |
| vsize (syn) | 0.350 | 0.250 | 0.180 | 0.120 | 0.080 | 0.040 | 0.015 | 0.001 |
| Param | CL1 | CL2 | CL3 | CL4 | CL5 | CL6 | CL7 | CL8 |
| (real) | 0.800 | 0.600 | 0.400 | 0.300 | 0.200 | 0.120 | 0.060 | 0.004 |
| (syn) | 8.000 | 4.000 | 0.400 | 0.250 | 0.150 | 0.070 | 0.030 | 0.004 |
| Overall Quality | Geometry Quality | Color Quality | |||||||
| Methods / Metrics | SRCC | PLCC | KRCC | SRCC | PLCC | KRCC | SRCC | PLCC | KRCC |
| PSNR [ 13 ] | 0.5173 | 0.5042 | 0.3652 | 0.4976 | 0.4843 | 0.3316 | 0.4782 | 0.4691 | 0.3213 |
| SSIM [ 13 ] | 0.5391 | 0.5073 | 0.3845 | 0.5265 | 0.5076 | 0.3815 | 0.4845 | 0.4535 | 0.3364 |
| LPIPS [ 41 ] | 0.5448 | 0.4927 | 0.3770 | 0.5518 | 0.5033 | 0.3645 | 0.4683 | 0.4103 | 0.3024 |
| DISTS [ 42 ] | 0.7267 | 0.7246 | 0.5016 | 0.7109 | 0.6937 | 0.4976 | 0.6618 | 0.6403 | 0.4545 |
| VIF [ 43 ] | 0.4137 | 0.4228 | 0.2949 | 0.4003 | 0.4461 | 0.2842 | 0.3505 | 0.3537 | 0.2631 |
| Method | GSC-QA [ 14 ] | NeRF-VSQA [ 23 ] | ENeRF-QA [ 11 ] | ||||||
| SRCC | PLCC | KRCC | SRCC | PLCC | KRCC | SRCC | PLCC | KRCC | |
| DOVER [ 52 ] | 0.8174 | 0.7994 | 0.6257 | 0.6757 | 0.7281 | 0.5146 | 0.9138 | 0.8991 | 0.7525 |
| FAST-VQA [ 28 ] | 0.6066 | 0.5741 | 0.4427 | 0.4175 | 0.4922 | 0.3019 | 0.7842 | 0.7796 | 0.5996 |
| simpleVQA [ 27 ] | 0.7307 | 0.7196 | 0.5339 | 0.2071 | 0.1770 | 0.1520 | 0.7281 | 0.7611 | 0.5390 |
| VSFA [ 26 ] | 0.7159 | 0.7101 | 0.5429 | 0.5517 | 0.5707 | 0.4043 | 0.9007 | 0.8862 | 0.7288 |
| 3DGSV-Assessor (NR) | 0.9130 | 0.8950 | 0.7539 | 0.7437 | 0.7796 | 0.5647 | 0.9471 | 0.9296 | 0.8249 |
| Component & Strategy | Overall Quality | Geometry Quality | Color Quality | ||||||||||
| No. | LMM | Ref-Cue | LoRA(vis) | LoRA(llm) | SRCC | PLCC | KRCC | SRCC | PLCC | KRCC | SRCC | PLCC | KRCC |
| (1) | ✔ | 0.8951 | 0.9029 | 0.7171 | 0.8391 | 0.8339 | 0.6448 | 0.8643 | 0.8666 | 0.6764 | |||
| (2) | ✔ | ✔ | 0.9008 | 0.9012 | 0.7195 | 0.8300 | 0.8448 | 0.6382 | 0.8885 | 0.8864 | 0.6985 | ||
| (3) | ✔ | ✔ | ✔ | 0.9617 | 0.9628 | 0.8367 | 0.9604 | 0.9628 | 0.8316 | 0.9590 | 0.9615 | 0.8211 | |
| (4) | ✔ | ✔ | 0.9489 | 0.9519 | 0.8216 | 0.9502 | 0.9571 | 0.8235 | 0.9451 | 0.9491 | 0.8076 | ||
| (5) | ✔ | ✔ | 0.9554 | 0.9552 | 0.8251 | 0.9472 | 0.9496 | 0.8195 | 0.9478 | 0.9523 | 0.8107 | ||
| Stage 2 (cue) | Overall | Geometry | Color | |||||||
| Intra-RankAcc | SRCC | Global-RankAcc | Global-SRCC | SRCC | PLCC | SRCC | PLCC | SRCC | PLCC | |
| K=1 (ours) | 0.9683 | 0.9295 | 0.9200 | 0.8895 | 0.9714 | 0.9726 | 0.9691 | 0.9727 | 0.9663 | 0.9703 |
| K=2 | 0.9619 | 0.9247 | 0.8616 | 0.7879 | 0.9702 | 0.9720 | 0.9678 | 0.9719 | 0.9655 | 0.9697 |
| K=3 | 0.9641 | 0.9362 | 0.8159 | 0.7008 | 0.9701 | 0.9718 | 0.9676 | 0.9717 | 0.9655 | 0.9695 |
| K=4 | 0.9565 | 0.9259 | 0.9037 | 0.8679 | 0.9700 | 0.9717 | 0.9676 | 0.9716 | 0.9654 | 0.9696 |
| Variant | Stage 2 (cue) | Overall | Geometry | Color | ||||||
| Intra-RankAcc | SRCC | Global-RankAcc | Global-SRCC | SRCC | PLCC | SRCC | PLCC | SRCC | PLCC | |
| Intra-scene ranking (ours) | 0.9683 | 0.9295 | 0.9200 | 0.8895 | 0.9714 | 0.9726 | 0.9691 | 0.9727 | 0.9663 | 0.9703 |
| Global ranking | 0.9625 | 0.9299 | 0.9432 | 0.9213 | 0.9689 | 0.9710 | 0.9689 | 0.9722 | 0.9651 | 0.9690 |
| Intra-scene ranking + cross-NCE | 0.9087 | 0.8445 | 0.6772 | 0.4211 | 0.9686 | 0.9708 | 0.9680 | 0.9724 | 0.9649 | 0.9682 |
| Global ranking + cross-NCE | 0.9504 | 0.9165 | 0.8890 | 0.8514 | 0.9676 | 0.9697 | 0.9674 | 0.9720 | 0.9656 | 0.9685 |
| Variant | Fusion | Heads | Overall | Geometry | Color | ||||||
| SRCC | PLCC | KRCC | SRCC | PLCC | KRCC | SRCC | PLCC | KRCC | |||
| Stage-1 (NR) | frozen | 0.9617 | 0.9628 | 0.8367 | 0.9604 | 0.9628 | 0.8316 | 0.9590 | 0.9615 | 0.8211 | |
| Concatenate cue | retrained | 0.9618 | 0.9669 | 0.8306 | 0.9666 | 0.9709 | 0.8389 | 0.9584 | 0.9673 | 0.8210 | |
| Sigmoid gate | frozen | 0.9688 | 0.9702 | 0.8469 | 0.9680 | 0.9711 | 0.8430 | 0.9641 | 0.9694 | 0.8347 | |
| Bounded gate (ours) | frozen | 0.9714 | 0.9726 | 0.8529 | 0.9691 | 0.9727 | 0.8461 | 0.9663 | 0.9703 | 0.8385 | |