cs.DCMar 21, 2026

Modernizing Amdahl's Law: How AI Scaling Laws Shape Computer Architecture

Authors: Chien-Ping Lu

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Classical Amdahl's Law conceptualized the limit of speedup for an era of fixed serial-parallel decomposition and homogeneous replication. Modern heterogeneous systems need a different conceptual framework: constrained resources must be allocated across heterogeneous hardware while workloads themselves change, with some stages becoming effectively bounded and others continuing to absorb additional effective compute. This paper reformulates Amdahl's Law around that shift. We replace processor count with an allocation variable, replace the classical parallel fraction with a value-scalable fraction, and model specialization by a relative efficiency ratio between dedicated and programmable compute. The resulting objective yields a finite collapse threshold. For a specialized efficiency ratio R, there is a critical scalable fraction S_c = 1 - 1/R beyond which the optimal allocation to specialization becomes zero. Equivalently, for a given scalable fraction S, the minimum efficiency ratio required to justify specialization is R_c = 1/(1-S). Thus, as value-scalable workload grows, over-customization faces a rising bar. The point is not that one hardware class simply defeats another, but that architecture must preserve a sufficiently programmable substrate against a moving frontier of work whose marginal gains keep scaling. In practice, that frontier is often sustained by software- and model-driven efficiency doublings rather than by fixed-function redesign alone. The model helps explain the migration of value-producing work toward learned late-stage computation and the shared design pressure that is making both GPUs and AI accelerators more programmable2

Figures & tables

AspectCPU programmabilityGraphics programmabilityAI programmability
What the user seesNew application functionsBetter visual quality and richer effectsMore capable models and better responses
Why it mattersOne machine must support many tasksSoftware can replace fixed stages with richer onesSoftware can keep improving models between redesigns
Architectural meaningGeneral instruction substrateShared graphics-compute substrateShared tensor-memory-interconnect substrate
Table 1: Programmability is a shifting architectural concept. What counts as “programmable” changes with the dominant workload and with the kind of improvement that software can continue to unlock on existing hardware.
Figure 1: Historical legacy of classical scaling laws. Amdahl’s law (solid) and Gustafson’s law (dashed) shown side by side in speedup form and normalized-time form. Both are expressed in terms of processor count NN and replication-based speedup, which is the framing the present paper leaves behind.
Figure 2: Normalized execution time T(x)T(x) versus specialization fraction xx for R=10R=10 and varying SS . Dashed markers indicate the optimal allocation x∗x^{*} . For low SS , the curves are U-shaped and specialization is beneficial; as SS approaches Sc=0.9S_{c}=0.9 , the optimum collapses toward the origin. The dashed black curve traces the optimal locus, terminating at the collapse point x∗=0x^{*}=0 . Above the threshold ( S=0.95S=0.95 ), the curve is monotonically increasing and no investment in dedicated hardware is optimal.
Figure 3: Threshold diagram in (S, R)(S,\,R) space for the optimal specialization boundary. The curve Rc=1/(1−S)R_{c}=1/(1-S) marks the minimum efficiency ratio required to justify a nonzero specialized allocation. As SS rises, the efficiency bar for more narrowly specialized mechanisms rises with it. Above the curve, a nonzero specialized allocation reduces total execution time; below it, the optimal specialized allocation falls to x∗=0x^{*}=0 .
Figure 4: Example rendered images illustrating how neural denoising and reconstruction shift graphics workload structure. Low-sample Monte Carlo rendering provides a noisy acquisition signal, while learned denoising recovers useful image quality from that input; as reconstruction quality improves, brute-force Monte Carlo rendering becomes effectively value-bounded and a larger share of the scalable workload shifts into learned post-processing. The example shown is the Crytek Sponza scene at 16 samples per pixel from the Intel Open Image Denoise gallery.
Figure 5: Shift of graphics workload structure under rising SS . In ray-traced or path-traced rendering with learned reconstruction, neural denoising and reconstruction compress the classical high-sample rendering regime: once useful image quality can be recovered from low-resolution or low-sample acquisition, brute-force Monte Carlo rendering becomes effectively value-bounded, primary and secondary visibility increasingly behave as bounded acquisition stages, and passes 3+ absorb a larger share of the scalable workload through anti-aliasing, denoising, reconstruction, frame synthesis, and related post-processing. In the extreme limit, many of these later passes collapse into a single learned reconstruction stage.
More in this topic

Heterogeneous Computing

Heterogeneous computing studies how to coordinate different processors, accelerators, and memory systems to improve performance, efficiency, and cost.

See all 80 papers

Explore similar work

CardsList
Mar 30, 2026

Continued AI Scaling Requires Repeated Efficiency Doublings

This paper argues that continued AI scaling requires repeated efficiency doublings. Classical AI scaling laws remain useful because they make progress predictable despite diminishing returns, but the compute variable in those laws is best read as logical compute, not as a record of one fixed physical implementation. Practical burden therefore depends on the efficiency with which physical resources realize that compute. Under that interpretation, diminishing returns mean rising operational burden, not merely a flatter curve. Sustained progress then requires recurrent gains in hardware, algorithms, and systems that keep additional logical compute feasible at acceptable cost. The relevant analogy is Moore's Law, understood less as a theorem than as an organizing expectation of repeated efficiency improvement. AI does not yet have a single agreed cadence for such gains, but recent evidence suggests trends that are at least Moore-like and sometimes faster. The paper's claim is therefore simple: if AI scaling is to remain active, repeated efficiency doublings are not optional. They are required.

Chien-Ping Lu

AI Scaling Laws
AskPDF
May 17, 2026

Active Budget Allocation for Efficient Scaling Law Estimation via Surrogate-Guided Pruning

Predicting model performance at larger scales enables the design of training strategies and architectures tailored to specific performance targets. Empirical scaling law research identifies functional forms to aid this prediction task. These describe the relationship between loss and compute using a loss-compute frontier defined by learning curves. Due to the empirical nature of this approach, the computational burden is substantial, making strategic resource allocation essential - yet it remains surprisingly underexplored. In this work, we address this shortcoming by exploring the suitability of Successive Halving (SH) and SH combined with parametric and non-parametric surrogate models. In addition to enabling a more systematic allocation of a given compute budget, our findings show that SH paired with surrogate models yields a set of learning curves that includes one with a lower loss-compute value than what naive uniform allocation or an SH-only approach can obtain. Our experiments demonstrate mean relative improvements of up to 2.84% and 5.47% on real-world and synthetic learning curve datasets. This strategic resource allocation enables us to obtain accurate scaling laws at significantly reduced computational costs, saving up to 98.7% over the traditional exhaustive approach.

Viktoria Schram, Markus Hiller, Daniel Beck +1

The School of Computing and Information Systems, the University of Melbourne, Melbourne, Australia · Royal Melbourne Institute of Technology, Melbourne, Australia · Now at Google Australia.

Multi-Fidelity OptimizationScaling LawsLearning Curves
AskPDF
Sep 4, 2026

Amortizing Scaling Law Construction Costs

Scaling laws guide the design choices for training large foundation models, but deriving them involves training an exhaustive grid over hyperparameters, token budgets, and parameter counts, which is computationally expensive. Fitting a scaling law, however, only requires the best-loss frontier across compute scales, discarding most of the trained configurations. We propose a framework for efficient scaling law construction that formulates data collection as a Bayesian optimization problem, and introduce metrics for comparing scaling law fitting methods under constrained compute budgets. We find that progressively expanding the compute budget during acquisition, mirroring the compute-ordered evaluation of configurations in practice, substantially improves recovery efficiency. Augmenting the observed configurations with surrogate-fantasized evaluations then recovers the broader experimental grid, allowing accurate scaling law fitting without training every configuration. Together, these can closely match scaling law fits over a full dense grid at computational savings of up to 10–100×10\text{--}100\times.

Abhash Kumar Jha, Diana Alexandra Onuţu, Neeratyoy Mallik +6

ELLIS Institute Tübingen · OpenEuroLLM · Eindhoven University of Technology +4

Language Model Scaling LawsBayesian OptimizationHyperparameter Optimization
AskPDF