Continuous Adversarial Flow Models
Organizations: ByteDance Seed
Abstract
We propose continuous adversarial flow models, a type of continuous-time flow model trained with an adversarial objective. Unlike flow matching, which uses a fixed mean-squared-error criterion, our approach introduces a learned discriminator to guide training. This change in objective induces a different generalized distribution, which empirically produces samples that are better aligned with the target data distribution. Our method is primarily proposed for post-training existing flow-matching models, although it can also train models from scratch. On the ImageNet 256px generation task, our post-training substantially improves the guidance-free FID of latent-space SiT from 8.26 to 3.63 and of pixel-space JiT from 7.17 to 3.57. It also improves guided generation, reducing FID from 2.06 to 1.53 for SiT and from 1.86 to 1.80 for JiT. We further evaluate our approach on text-to-image generation, where it achieves improved results on both the GenEval and DPG benchmarks.
Figures & tables
| Method | PE | CFG | Single Obj. | Two Obj. | Color Attr. | Position | Counting | Colors. | Overall |
|---|---|---|---|---|---|---|---|---|---|
| FM | No | No | 0.72 | 0.23 | 0.11 | 0.09 | 0.25 | 0.59 | 0.33 |
| CAFM | 0.85 | 0.42 | 0.17 | 0.16 | 0.41 | 0.61 | 0.44 | ||
| FM | Yes | No | 0.95 | 0.66 | 0.35 | 0.40 | 0.42 | 0.81 | 0.60 |
| CAFM | 0.99 | 0.83 | 0.50 | 0.52 | 0.57 | 0.86 | 0.71 | ||
| FM | Yes | Yes | 0.99 | 0.89 | 0.62 | 0.69 | 0.77 | 0.89 | 0.81 |
| CAFM | 0.99 | 0.92 | 0.71 | 0.71 | 0.81 | 0.94 | 0.85 |
| Method | CFG | Global | Entity | Attribute | Relation | Other | Overall |
|---|---|---|---|---|---|---|---|
| FM | No | 81.34 | 82.96 | 81.71 | 83.17 | 85.07 | 72.25 |
| CAFM | 87.82 | 86.65 | 86.33 | 86.49 | 84.85 | 77.21 | |
| FM | Yes | 90.34 | 90.56 | 88.98 | 88.17 | 90.71 | 83.67 |
| CAFM | 89.55 | 89.83 | 89.99 | 91.20 | 91.88 | 85.21 |
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| CFG | Method | Epoch | FID | IS | sFID | Prec. | Rec |
|---|---|---|---|---|---|---|---|
| None | SiT | 1400 | 8.26 | 131.65 | 6.32 | 0.68 | 0.67 |
| SiT+FM | 1400+10 | 8.64 | 131.91 | 6.36 | 0.68 | 0.67 | |
| SiT+CAFM | 1400+10 | 3.63 | 178.08 | 4.72 | 0.71 | 0.69 | |
| 1.1 | SiT | 1400 | 5.55 | 161.77 | 5.67 | 0.72 | 0.66 |
| SiT+FM | 1400+10 | 5.53 | 161.72 | 5.68 | 0.72 | 0.65 | |
| SiT+CAFM | 1400+10 | 2.27 | 212.06 | 4.55 | 0.74 | 0.67 |
| CFG | Epoch | FID | IS | sFID | Prec. | Rec | |
|---|---|---|---|---|---|---|---|
| None | Non-saturating | 1400+10 | 3.54 | 167.45 | 5.20 | 0.72 | 0.68 |
| Hinge | 1400+10 | 4.00 | 175.21 | 5.36 | 0.71 | 0.68 | |
| Least squares | 1400+10 | 3.63 | 178.08 | 4.72 | 0.71 | 0.69 | |
| 1.3 | Non-saturating | 1400+10 | 1.58 | 245.10 | 4.84 | 0.79 | 0.64 |
| Hinge | 1400+10 | 1.57 | 258.46 | 4.86 | 0.79 | 0.64 | |
| Least squares | 1400+10 | 1.53 | 263.52 | 4.59 | 0.78 | 0.64 |
| CFG | Method | Epoch | FID | IS | sFID | Prec. | Rec |
|---|---|---|---|---|---|---|---|
| None | JiT | 600 | 7.17 | 151.54 | 5.51 | 0.68 | 0.67 |
| JiT+FM | 600+10 | 9.30 | 139.00 | 6.16 | 0.67 | 0.66 | |
| JiT+CAFM | 600+10 | 3.57 | 198.08 | 4.77 | 0.74 | 0.65 | |
| 1.2 | JiT | 600 | 4.60 | 188.88 | 5.34 | 0.71 | 0.66 |
| JiT+FM | 600+10 | 5.97 | 176.99 | 5.88 | 0.71 | 0.65 | |
| JiT+CAFM | 600+10 | 2.47 | 232.69 | 4.76 | 0.76 | 0.64 |
| SiT | JiT | |
| learning rate | 1e-5 | 1e-5 |
| batch size | 256 | 1024 |
| total epoch | 10 | 10 |
| warm-up epoch | 2 | 4 |
| timesteps | uniform(0, 1) | lognormal(0.8, 0.8)* |
| CFG interval | [0, 1] | [0, 0.9]* |
| FM | CAFM | |
| learning rate | 5e-5 | 5e-5 |
| learning rate | 3e-5 | |
| 16 | ||
| 0 | ||
| AdamW | (0.9, 0.95) | (0, 0.95) |
| iterations | 30k | FM10k+20k |
| Method | PE | CFG | Single Obj. | Two Obj. | Color Attr. | Position | Counting | Colors. | Overall |
|---|---|---|---|---|---|---|---|---|---|
| ZImage | No | No | 0.78 | 0.37 | 0.16 | 0.11 | 0.32 | 0.59 | 0.39* |
| ZImage+FM | 0.72 | 0.23 | 0.11 | 0.09 | 0.25 | 0.59 | 0.33 | ||
| ZImage+CAFM | 0.85 | 0.42 | 0.17 | 0.16 | 0.41 | 0.61 | 0.44 | ||
| ZImage | Yes | No | 0.96 | 0.83 | 0.43 | 0.47 | 0.51 | 0.85 | 0.68* |
| ZImage+FM | 0.95 | 0.66 | 0.35 | 0.40 | 0.42 | 0.81 | 0.60 | ||
| ZImage+CAFM | 0.99 | 0.83 | 0.50 | 0.52 | 0.57 | 0.86 | 0.71 |
| Method | CFG | Global | Entity | Attribute | Relation | Other | Overall |
|---|---|---|---|---|---|---|---|
| ZImage | No | 87.27 | 87.22 | 86.44 | 88.97 | 89.21 | 79.83* |
| ZImage+FM | 81.34 | 82.96 | 81.71 | 83.17 | 85.07 | 72.25 | |
| ZImage+CAFM | 87.82 | 86.65 | 86.33 | 86.49 | 84.85 | 77.21 | |
| ZImage | Yes | 90.55 | 91.71 | 91.49 | 92.43 | 87.94 | 86.35* |
| ZImage+FM | 90.34 | 90.56 | 88.98 | 88.17 | 90.71 | 83.67 | |
| ZImage+CAFM | 89.55 | 89.83 | 89.99 | 91.20 | 91.88 | 85.21 |
Flow-Based Generative Modeling
Flow-based generative modeling learns probability transports from simple noise to data for fast sampling, density estimation, and controllable generation.
See all 530 papers