Organizations: Centre for Electronics Frontiers, Institute for Integrated Micro and Nano Systems, School of Engineering, The University of Edinburgh, UK
Vision generative artificial intelligence (AI) has emerged as one of the most rapidly advancing areas of deep learning. The explosion of multimodal models has made them widely associated with text-to-image applications running on large datacentres. However, vision generative models are equally needed in applications that operate under strict hardware constraints at the edge, including autonomous vehicles, agricultural sensors, and mobile devices. In this Perspective, we argue that progress in vision generative AI has been driven by output quality, with hardware evolving reactively to accommodate growing model demands. We quantify the parameter cost and energy efficiency of these models across a range of accelerator platforms, and map four generative model families against seven real-world application domains. Finally, we advocate a software-hardware co-design approach, where deployment constraints are considered from the start of the design process, ensuring that the "right model" runs on the "right hardware" to serve the "right application", making generative AI deployment sustainable and accessible across a much broader range of platforms.
Figures & tables
Figure 1: Evolution and core architectures of vision generative models . a, Timeline of landmark generative models from 2013 to 2024, with each model represented as a point coloured by architectural family: Variational autoencoders (VAE; blue), generative adversarial networks (GAN; green), diffusion models (orange), and autoregressive transformer models (purple). b, VAE architecture, in which an encoder maps the input image to a probabilistic latent space and a decoder reconstructs the image from a sampled latent vector. c, GAN architecture, comprising a generator that synthesises images from random noise and a discriminator trained to distinguish generated from real images through an adversarial objective. d, Diffusion model architecture, in which an image is progressively corrupted by Gaussian noise during a forward process and a neural network learns to reverse this process iteratively to generate new samples. The arrow indicates the architectural evolution toward the diffusion transformer (DiT), in which the conventional U-Net backbone is replaced by a transformer e, Autoregressive transformer architecture, in which images are represented as sequences of discrete tokens and generated autoregressively using a transformer trained with a next-token prediction objective.
Figure 2: Evolution of AI hardware accelerators. Timeline of key hardware milestones from 2012 to 2025, each annotated with its key architectural contribution, coloured by architectural family: GPUs (blue), systolic arrays and TPUs (red), FPGAs (green), near-memory and compute-in-memory architectures (CIM; orange), and custom ASICs including neuromorphic processors (purple). The timeline reveals a reactive pattern: each wave of hardware innovation followed, rather than preceded, the model demands that motivated it.
Figure 3: Parameter efficiency and computational performance of generative vision models. a, Fréchet inception distance (FID; lower values indicate closer distributional alignment with real data) versus number of parameters for generative models across three benchmark datasets: CIFAR-10 (blue background, circle markers), ImageNet 128 × 128 (pink background, square markers), and ImageNet 256× 256 (yellow background, triangle markers). Generative adversarial networks (GANs), diffusion, and autoregressive (transformer) models are shown in green, orange, and purple, respectively. b, Computational throughput (TOPS) versus power consumption (watts) for representative model and hardware platform pairs. Diagonal dotted lines represent energy efficiency levels of 10 GOPS/W, 100 GOPS/W, 1 TOPS/W, 10 TOPS/W, 100 TOPS/W, and 1000 TOPS/W. Architecture families follow the same colour encoding as a . Hardware platforms are distinguished by marker shape: circles denote GPUs, triangles denote FPGAs, squares denote ASICs, and diamonds TPUs.
Figure 4: The right model for the right application. a, Compatibility matrix mapping the four generative model families, variational autoencoders (VAEs; blue), generative adversarial networks (GANs; green), diffusion models (orange), and autoregressive transformer models (purple), against seven application domains. Filled circles indicate architectural feasibility within the deployment constraints of each domain. b, Fréchet inception distance (FID) score upper limits for quality-constrained application domains, overlaid with the FID ranges of the four model families. Applications whose feasibility is primarily determined by output quality are shown. c, Inference latency upper limits for latency-constrained application domains, overlaid with the latency ranges of the four model families. Applications whose feasibility is primarily determined by inference time are shown. d, Memory footprint upper limits for memory-constrained application domains, overlaid with the memory ranges of the four model families. Applications whose feasibility is primarily determined by memory capacity are shown. Together, panels b, c, and d show where each model family falls relative to the maximum acceptable value for each application.
Figure 5: Three possible trajectories for the deployment of efficient generative AI. a, The first trajectory describes the present moment, in which models improve in accuracy alongside growing datacenters infrastructure and energy consumption. b, The second trajectory represents the software-hardware co-design approach advocated in this Perspective, in which models are matched to the right hardware to serve the right application. Some applications run locally on edge devices, but dependence on remote datacenters persists. c, The third trajectory describes the ambient future. Post-CMOS computing architectures and algorithmic reconsiderations at the foundational AI level enable applications to run entirely on edge devices effectively, eliminating the need for large datacenters and making intelligence accessible everywhere.
Category
Model / Application
FID ( ↓ )
Inference Time ( ↓ )
DRAM (GB)
Model families
VAE
28–32
< 100 ms
< 4
GAN
2.3–8
30–70 ms
< 8
Diffusion
2.1–11
0.8–2.5 s
∼ 10
Transformer
1.5–15
2–10 s
>24
Application requirements
Quality-constrained
Cloud content gen.
< 5 ⋆
—
—
Table S1: Model metrics and application deployment requirements corresponding to Figures 4 a–d. Inference time and DRAM ranges for each model family are reported from [ 70 ] . FID scores are evaluated on the ImageNet 256 × 256 dataset. Inference time is measured for a single 512 × 512 image on an NVIDIA A100 GPU. For each application, only the primary binding constraint ( ⋆ ) is reported. FID thresholds for cloud content generation represent perceptual sufficiency bounds defined by the authors.
Vision-Language-Action (VLA) models are an emerging class of workloads critical for robotics and embodied AI at the edge. As these models scale, they demonstrate significant capability gains, yet they must be deployed locally to meet the strict latency requirements of real-time applications. This paper characterizes VLA performance on two generations of edge hardware, viz. the Nvidia Jetson Orin and Thor platforms. Using MolmoAct-7B, a state-of-the-art VLA model, we identify a primary execution bottleneck: up to 75% of end-to-end latency is consumed by the memory-bound action-generation phase. Through analytical modeling and simulations, we project the hardware requirements for scaling to 100B parameter models. We also explore the impact of high-bandwidth memory technologies and processing-in-memory (PIM) as promising future pathways in edge systems for embodied AI.
Generative Artificial Intelligence (GenAI) features such as image editing, object removal, and prompt-guided image transformation are increasingly integrated into mobile applications. However, deploying Large Vision Models (LVMs) for such tasks on resource-constrained devices remains challenging due to their high memory and compute requirements. While Low-Rank Adapters (LoRAs) enable parameter-efficient task adaptation, existing Mobile deployment pipelines typically compile separate model binaries for each LoRA + a copy of the foundation model, resulting in redundant storage and increased runtime overhead. In this work, we present a unified framework for enabling multi-task GenAI inference on edge devices using a single shared model. Our key idea is to treat LoRA weights as runtime inputs rather than embedding them into the compiled model graph, allowing dynamic task switching at runtime without recompilation. Then, to support efficient on-device execution, we introduce QUAD (Quantization with Unified Adaptive Distillation), a quantizationaware training strategy that aligns multiple LoRA adapters under a shared quantization profile. We implement the proposed system with a lightweight runtime stack compatible with mobile NPUs and evaluate it across multiple chipsets. Experimental results demonstrate up to 6x and 4x reduction in memory footprint and latency improvements, respectively, while maintaining high visual quality across multiple GenAI tasks.
The incorporation of generative models into traditional computational systems presents both enormous opportunity and tremendous peril. Although many early adopters have realized these perils at great expense, the field still requires foundational frameworks to de-risk incorporation of AI into traditional systems. This manuscript establishes this foundation through the definition of four specific primitives of AI blended architecture, designed to enable deterministic encapsulation of probabilistic models. It further establishes two overarching anti-patterns broadly represented across industry to serve as warnings for engineers in this field. This framework was designed to enable successful integration of AI into traditional systems while providing a foundation upon which generative model providers could build the next generation of generative model interfaces.