FastGRN: Few-Step Generative Refinement Networks
Organizations: Ritsumeikan University · Osaka Institute of Technology
Abstract
Generative Refinement Networks (GRN) generate an image by repeatedly re-predicting and refining every token of a discrete latent. They reach high fidelity, but need dozens of Markov steps. We therefore train few-step GRNs on the distribution of their own samples. We use its own uniform-state corruption to perform distribution matching on discrete tokens, which lets a discrete generator learn by backpropagation, and we pair noise with samples in the manner of rectified flow. We find that the steps of a few-step GRN do not carry the same amount of information or the same repair load. We therefore exploit its discrete refinement during training and build few-step GRNs in three parts. Oversampling samples the student with more steps than it was trained for and refines its judge on these samples. Non-uniform sampling trains only the last step and hands it most of the information at sampling time. Refinement confidence keeping holds on to the tokens the student is already sure of. We train a fast class-conditional generator on ImageNet 256 and, with the same method, a 2B text-to-image model. On ImageNet 256, the 3-step FastGRN beats the FID of GRN. On a 2B binary-token text-to-image GRN, a 4-step student also nearly matches the GenEval of GRN-T2I 2B.
Figures & tables
| Method | Steps | NFE | ms | FID | T. | IS |
| GRN-B (130M) | ||||||
| Teacher | 50 | 79 | 835 | 3.73 | – | 305.9 |
| GRN-B | 8 | 12 | – | 8.13 | 2.18 | 189.8 |
| Fine-tuned GRN | 8 | 8 | – | 14.0 | 3.76 | 151.0 |
| Paired-noise GRN | 2 | 2 | 66 | 58.2 | 15.60 | 47.2 |
| Matching without window | 2 | 2 | 66 | 5.39 | 1.45 | 244.4 |
| Method | Teacher | Params | NFE | FID | T. |
| Di[M]O [ 110 ] | MaskGIT | 174 | 1 | 6.91 | 1.05 |
| FPD [ 88 ] | MaskGIT | 174 † | 1 | 6.90 | 1.05 |
| Soft-Di[M]O g [ 111 ] | MaskGIT | 174 † | 1 | 6.40 | 0.97 |
| Di[M]O [ 110 ] [ 111 ] | MaskBit | 305 | 1 | 2.89 | 1.74 |
| Soft-Di[M]O g [ 111 ] | MaskBit | 305 | 1 | 1.96 | 1.18 |
| Soft-Di[M]O g [ 111 ] (longer) | MaskBit | 305 | 1 | 1.56 | 0.94 |
| Method | Teacher | Params | Steps | GenEval | T. |
| Di[M]O [ 110 ] | MaskGen-L | 0.6B | 1 | 0.42 | 0.88 |
| FPD [ 88 ] | MaskGen-L | 0.6B | 1 | 0.45 | 0.94 |
| Soft-Di[M]O [ 111 ] | MaskGen-L | 0.6B | 1 | 0.51 | 1.06 |
| Soft-Di[M]O [ 111 ] | MaskGen-L | 0.6B | 1 | 0.63 | 1.31 |
| Show-o Turbo [ 98 ] ( ) | Show-o | 1.3B | 2 | 0.49 | 0.84 |
| Show-o Turbo [ 98 ] ( ) | Show-o | 1.3B | 4 | 0.52 | 0.88 |
| at iteration | 3k | 3k at | 3k at (50k) | ||||||||
| 2-step student continued with | 1k | 2k | 3k | P T | R T | FID-T | IS | FID | IS | ||
| Before (10k student) ( W ) | 6.65 (start) | 6.04 | 6.23 | 0.692 | 0.783 | 4.11 | – | 4.17 | 277.6 | ||
| Original judge ( W ) | 6.50 | 6.51 | 6.50 | 6.19 | – | 0.700 | 0.784 | 3.97 | 216.6 | 4.12 | 280.4 |
| Teacher-data judge ( W ) | 6.51 | 6.58 | 6.58 | 6.45 | – | 0.704 | 0.783 | 3.92 | 224.3 | – | – |
| Oversampling judge ( W + OS ) | 6.25 | 6.22 | 6.25 | 6.22 | – | 0.705 | 0.773 | 3.92 | 233.9 | – | – |
| Oversampling judge (student of Tab. 1 ) ( W + OS ) | 6.29 | 6.21 | 6.24 | 6.25 | 6.52 | – | – | – | 234.1 | 3.83 | 306.8 |
| 0.05 | 0.1 | 0.2 | 0.3 | 0.4 | 0.6 | 0.8 | |
| 0.0019 | 0.0013 | 0.0026 | 0.0032 | 0.0031 | 0.0030 | 0.0028 | |
| 0.673 | 0.689 | 0.711 | 0.631 | 0.551 | 0.398 | 0.245 |
| (a) 2-step GRN-L students (FID 50k ) | |||||
| Student (grid point ) | FID 50k | IS 50k | P T | R T | FID-T |
| Base (10k W ) on | 3.648 | 310.4 | 0.7394 | 0.6610 | 2.061 |
| +2k on | 3.834 | 313.7 | 0.7459 | 0.6523 | 2.254 |
| +2k on (NS) ( W + NS ) | 3.092 | 296.4 | 0.7158 | 0.7097 | 1.594 |
| NS sampled at | 3.243 | 289.2 | 0.7142 | 0.6987 | – |
| (b) Grid ablations (FID-10k unless noted; against the reference) | |||||
| (a) 2-step W + OS student (FID 50k ) | ||||
| Student (grid point ) | FID 50k | IS 50k | P T | R T |
| Base ( W + OS ) on | 3.826 | 306.1 | 0.712 | 0.708 |
| +2k on | 3.865 | 307.0 | 0.707 | 0.708 |
| +2k on (NS) ( W + OS + NS ) | 3.730 | 298.6 | 0.690 | 0.719 |
| same student sampled at | 3.637 | 297.3 | 0.704 | 0.710 |
| +2k on | 3.708 | 301.7 | 0.699 | 0.718 |
| Diversity | Last | |||||
| Student | draft | final | step | Kept | Chance | |
| GRN-B ( W ) | ||||||
| Base | 0.110 | 0.601 | 0.596 | 0.808 | 0.251 | |
| +2k on | 0.155 | 0.481 | 0.809 | 0.762 | 0.288 | |
| +2k on | 0.118 | 0.671 | 0.449 | 0.826 | 0.253 | |
| Base sampled at | 0.110 | 0.398 | 0.833 | 0.669 | 0.224 | |
| (a) GRN-B teacher (FID-10k) | |||||
| Sampling rule | Extra | 25 steps | Extra | 16 steps | 50 steps |
| Without keeping | 0% | 9.59 | 0% | 18.29 | 6.17 |
| Refinement confidence keeping | 12% | 6.63 | 9% | 8.33 | – |
| Freezing | 14% | 8.13 | 10% | 13.20 | – |
| Bit keeping | 9% | 6.67 | 5% | 11.15 | – |
| Bit freezing | 12% | 10.99 | 7% | 9.84 | – |
| Setting | FID | Best (it.) | Prec. | Rec. | Training |
| (a) Initialization objective | |||||
| Fine-tuned init. without window | 358.3 | 17.69 (8k) | 0.001 | 0.045 | collapse 7.9k |
| Paired-noise init. without window | 332.8 | 7.87 (4k) | 0.020 | 0.442 | collapse 6.4k |
| Fine-tuned init. with window | 9.85 | 9.85 (10k) | 0.632 | 0.758 | stable |
| Paired-noise init. with window | 6.65 | 6.59 (6k) | 0.692 | 0.783 | stable |
| (b) Objective variants (FID-50k) | |||||
| Sampling steps | 2 | 4 | 8 | 16 |
| GRN-B (teacher at 79 NFE: 6.24) | ||||
| GRN-B with fewer steps n | 112.5 | 49.6 | 10.9 | – |
| Fine-tuned GRN | 156.9 | 68.3 | 16.7 | 9.56 |
| Paired-noise GRN | 60.8 | 26.9 | 13.0 | 8.68 |
| Matching without window (best before collapse) | 7.87 c | 6.97 | – | – |
| FastGRN ( W ) | 6.65 | 6.42 | – | – |
| Oracle posteriors | Learned | |||||
| Target | Eq. ( 2 ) | Heuristic | fake | |||
| Dirichlet | 4 | 3 | 1.045 | 0.0021 | 0.0856 | 0.0765 |
| Ising | 6 | 2 | 1.639 | 0.0001 | 0.0002 | 0.0247 |
| Prototypes | 4 | 3 | – | 0.0000 | 0.0002 | 0.0191 |
| Parity | 4 | 2 | 0.495 | 0.0000 | 0.4947 | 0.0186 |
| GenEval (553 prompts 4 images) | 1k COCO captions | ||||||||||||||
| Method | Iter. | NFE | Overall | Single | Two | Count | Color | Pos. | Attr. | vs. T. | FID-T | CLIP | HPS | Div. | |
| GRN-T2I [ 29 ] ( ) ∗ | – | – | 100 | 0.76 | 0.99 | 0.90 | 0.72 | 0.84 | 0.52 | 0.60 | – | – | – | – | – |
| Teacher (50 steps at ) | – | – | 100 | 0.695 | 0.98 | 0.83 | 0.71 | 0.79 | 0.33 | 0.53 | – | 36.6 a | 26.58 | 28.19 | 1.00 |
| GRN-T2I (16 steps) | – | – | 32 | 0.687 | 0.98 | 0.82 | 0.69 | 0.79 | 0.31 | 0.53 | 0.008 | 34.4 | 26.61 | 27.42 | – |
| GRN-T2I (8 steps) | – | – | 16 | 0.563 | 0.93 | 0.63 | 0.52 | 0.69 | 0.24 | 0.37 | 0.132 | 54.5 | 26.18 | 25.07 | – |
| GRN-T2I (4 steps) | – | – | 8 | 0.209 | 0.54 | 0.11 | 0.14 | 0.35 | 0.06 | 0.05 | – | 141.3 | 22.51 | 19.41 | – |
| Method | Steps | NFE | FID | T. | IS | P T | R T | ms |
| GRN-B (130M) | ||||||||
| Teacher | 50 | 79 | 3.73 | – | 305.9 | 0.733 | 0.730 | 835 |
| GRN-B | 16 | 25 | 3.37 | 0.90 | 287.2 | 0.690 | 0.719 | – |
| GRN-B | 14 | 22 | 3.51 | 0.94 | 282.5 | 0.681 | 0.720 | – |
| GRN-B | 12 | 19 | 3.60 | 0.96 | 278.0 | 0.668 | 0.702 | – |
| GRN-B | 10 | 15 | 4.74 | 1.27 | 228.5 | 0.593 | 0.711 | – |
| Method | Base | Params | NFE | FID | T. | IS |
| Further discrete students | ||||||
| SDTT [ 18 ] [ 102 ] | MaskGIT | 174 † | 1 c | 90.40 | – | 14 |
| ReDi [ 102 ] | MaskGIT | 174 † | 1 c | 37.43 | – | 49 |
| ReDi [ 102 ] (2 rounds) | MaskGIT | 174 † | 1 c | 21.80 | – | 90 |
| ReDi [ 102 ] (2 rounds) | MaskGIT | 174 † | 4 c | 7.86 | – | 240 |
| ReDi-distill [ 102 ] | MaskGIT | 174 † | 1 c | 11.68 | – | 182 |
| Method | Teacher | Params | Steps | GenEval |
| Continuous | ||||
| InstaFlow [ 53 ] | – | 0.9B | 1 | 0.33 |
| SANA-Sprint g [ 13 ] | – | 0.6B | 1 | 0.72 |
| SANA-Sprint g [ 13 ] | – | 0.6B | 2 | 0.76 |
| SANA-Sprint g [ 13 ] | – | 0.6B | 4 | 0.76 |
| SANA-Sprint g [ 13 ] | – | 1.6B | 1 | 0.76 |
| Setting | GenEval | FID-T | HPS | Div. | Healthy |
| (a) Initialization at (1k it.) | |||||
| Fine-tuned init. | – | 48.6 | 25.51 | 0.73 | no |
| Paired-noise init. | – | 42.1 | 26.16 | 0.90 | yes |
| (b) Real score at (2k it.) | |||||
| Guided teacher | 0.602 | 49.9 | 26.11 | 0.85 | yes |
| Fine-tuned | 0.637 | 41.0 | 26.38 | 0.90 | yes |
| Large batch | Batch 1 (4090) | ||||||
| Model | Sampler | NFE | ms/img | Net | +Dec. | ms | Gain |
| GRN-B | Teacher | 79 | 34.9 | 1.0 | 1.0 | 835 | 1.0 |
| FastGRN | 4 | 1.89 | 18.4 | 2.5 | 94 | 8.9 | |
| FastGRN | 2 | 0.97 | 35.8 | 2.6 | 66 | 12.7 | |
| GRN-L | Teacher | 79 | 107.3 | 1.0 | 1.0 | 1292 | 1.0 |
| FastGRN | 4 | 5.62 | 19.1 | 5.3 | 111 | 11.7 | |