Refining Few-Step Text-to-Multiview Diffusion via Reinforcement Learning
Organizations: School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University · School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University · Tencent Inc. · Xiaomi Inc., China
Abstract
Text-to-multiview (T2MV) diffusion models have shown great promise in generating multiple views of a scene from a single text prompt. While few-step backbones enable real-time T2MV generation, they often compromise key aspects of generation quality, such as per-view fidelity and cross-view consistency. Reinforcement learning (RL) finetuning offers a potential solution, yet existing approaches designed for single-image diffusion do not readily extend to the few-step T2MV setting, as they neglect cross-view coordination and suffer from weak learning signals in few-step regimes. To address this, we propose MVC-ZigAL, a tailored RL finetuning framework for few-step T2MV diffusion models. Specifically, its core insights are: (1) a new MDP formulation that jointly models all generated views and assesses their collective quality via a joint-view reward; (2) a novel advantage learning strategy that exploits the performance gains of a self-refinement sampling scheme over standard sampling, yielding stronger learning signals for effective RL finetuning; and (3) a unified RL framework that extends advantage learning with a Lagrangian dual formulation for multiview-constrained optimization, balancing single-view and joint-view objectives through adaptive primal-dual updates under a self-paced threshold curriculum that harmonizes exploration and constraint enforcement. Collectively, these designs enable robust and balanced RL finetuning for few-step T2MV diffusion models, yielding substantial gains in both per-view fidelity and cross-view consistency. Code is available at https://github.com/ZiyiZhang27/MVC-ZigAL.
Figures & tables
| Method | HyperScore | PickScore | |||
|---|---|---|---|---|---|
| Alignment | Geometry | Texture | Overall | ||
| Baseline | 7.60 | 7.55 | 6.46 | 7.23 | 0.196 |
| MV-PG | 8.49 | 9.11 | 7.21 | 8.39 | 0.203 |
| MV-DPO | 8.07 | 8.75 | 7.02 | 8.00 | 0.200 |
| MV-RDL | 8.90 | 9.87 | 7.80 | 9.03 | 0.203 |
| MV-ZigAL | 9.20 | 9.92 | 7.84 | 9.17 | 0.205 |
| Method | HyperScore | PickScore | HPSv2 | ImageReward | |||
|---|---|---|---|---|---|---|---|
| Alignment | Geometry | Texture | Overall | ||||
| Baseline | 6.69 | 6.97 | 6.54 | 6.67 | 0.204 | 0.252 | -0.846 |
| MV-ZigAL | 6.88 | 7.41 | 6.69 | 6.95 | 0.205 | 0.254 | -0.770 |
| WS-ZigAL ( ) | 6.22 | 6.81 | 5.98 | 6.25 | 0.220 | 0.265 | 0.020 |
| WS-ZigAL ( ) | 6.78 | 7.23 | 6.69 | 6.83 | 0.217 | 0.270 | 0.183 |
| WS-ZigAL ( ) | 6.87 | 7.19 | 6.47 | 6.68 | 0.215 | 0.262 | 0.006 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Standard Training | Large-Scale Training | SDXL Training |
| Number of views ( ) | 6 | 6 | 6 |
| Number of inference steps ( ) | 8 | 8 | 16 |
| Image height | 768 | 768 | 768 |
| Image width | 768 | 768 | 768 |
| Learning rate | 1e-4 | 5e-5 | 5e-5 |
| Optimizer type | AdamW | AdamW | AdamW |
| Method | NFE | HyperScore | ImageReward | HPSv2 | PickScore | |||
|---|---|---|---|---|---|---|---|---|
| Alignment | Geometry | Texture | Overall | |||||
| (a) Results of MATE-3D SOTA methods | ||||||||
| SJC [ 44 ] | 50 | 4.02 | 2.96 | 2.65 | 3.01 | - | - | - |
| Latent-NeRF [ 27 ] | 50 | 4.53 | 3.44 | 3.43 | 3.56 | - | - | - |
| 3DTopia [ 12 ] | 200 | 4.36 | 4.04 | 4.37 | 4.05 | - | - | - |
| TextMesh [ 41 ] | - | 4.47 | 4.25 | 4.10 | 4.12 | - | - | - |